2024-04-24T22:17:21.639 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:17:21.639 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127 --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/orig.config.yaml 2024-04-24T22:17:21.700 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:17:21.795 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672127' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPkQaaN0BXV3ZAGSbkyZY4ZF/x3JSCgZo4jTpjBUQdyBNWnOfcmQ81ZGqPR0y3riUPBlHiqViYXOMqj0OXyy7XQ= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbjTSRDsU55kpPrbvIFcS2/95v6Hr7V2h5HmrJ3FaSrRmKApc7j6f/Fmdy0tEA9xNmftB9vJdJ9cOoYrhwtCqg= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEl/Y2YCqwxFmdl2hIC8sfzd6dB4rqlaDvbych1VyMc8sXcK6Xmyki/EJqDtAsH7P5HfXiW9odQ5qVNu/BhrhM4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:17:21.796 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:17:21.810 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:17:21.810 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:17:21.811 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:17:21.836 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:17:21.836 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:17:21.836 INFO:teuthology.packaging:ref: None 2024-04-24T22:17:21.836 INFO:teuthology.packaging:tag: None 2024-04-24T22:17:21.836 INFO:teuthology.packaging:branch: squid 2024-04-24T22:17:21.836 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:17:21.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:17:22.170 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:17:22.172 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:17:22.179 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:17:22.179 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:17:22.185 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:17:22.206 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:17:22.213 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:17:22.237 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:11:48.939464', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPkQaaN0BXV3ZAGSbkyZY4ZF/x3JSCgZo4jTpjBUQdyBNWnOfcmQ81ZGqPR0y3riUPBlHiqViYXOMqj0OXyy7XQ='} 2024-04-24T22:17:22.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:11:48.940754', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbjTSRDsU55kpPrbvIFcS2/95v6Hr7V2h5HmrJ3FaSrRmKApc7j6f/Fmdy0tEA9xNmftB9vJdJ9cOoYrhwtCqg='} 2024-04-24T22:17:22.283 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:11:48.937955', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEl/Y2YCqwxFmdl2hIC8sfzd6dB4rqlaDvbych1VyMc8sXcK6Xmyki/EJqDtAsH7P5HfXiW9odQ5qVNu/BhrhM4='} 2024-04-24T22:17:22.283 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:17:22.290 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T22:17:22.290 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T22:17:22.290 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T22:17:22.291 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:17:22.420 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c418ce160>, signals=[15]) 2024-04-24T22:17:22.421 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:17:22.427 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:17:22.427 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:17:22.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:17:22.507 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-24T22:17:22.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:17:22.581 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-24T22:17:22.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:17:22.658 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:17:22.664 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-04-24T22:17:22.682 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-04-24T22:17:22.682 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:17:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:17:22.741 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:17:22.742 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-04-24T22:17:22.762 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-24T22:17:22.780 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-24T22:17:22.780 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-24T22:17:22.837 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-24T22:17:22.837 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-24T22:17:22.837 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-24T22:17:22.837 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:17:22.838 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:17:22.838 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-24T22:17:22.862 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-24T22:17:22.881 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-24T22:17:22.881 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-24T22:17:22.939 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-24T22:17:22.939 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-24T22:17:22.939 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-24T22:17:22.939 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:17:22.941 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:17:22.942 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:17:22.942 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:17:22.942 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-24T22:17:22.962 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:17:22.972 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:17:22.998 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:17:22.998 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:17:23.000 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:17:23.003 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:17:23.019 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:17:23.025 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:17:23.025 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:17:23.057 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:17:23.060 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:17:23.166 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:17:23.242 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:17:23.257 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:17:23.257 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-24T22:17:23.257 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-24T22:17:23.257 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:17:23.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:17:23.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-24T22:17:23.510 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:17:23.510 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T22:17:23.510 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-04-24T22:17:23.527 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:17:23.528 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:17:23.528 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T22:17:23.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum upgrade -y linux-firmware 2024-04-24T22:17:25.832 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T22:17:29.341 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-04-24T22:17:34.292 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.2 MB 00:01 2024-04-24T22:17:36.285 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-24T22:17:40.194 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 21 MB 00:03 2024-04-24T22:17:45.185 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-24T22:17:47.118 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-24T22:17:47.119 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-04-24T22:17:47.119 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-24T22:17:47.240 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T22:17:47.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:17:47.388 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T22:17:47.389 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T22:17:47.391 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi111.front.sepia.ceph.com to complete... 2024-04-24T22:18:00.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24 22:18:00 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T22:18:00.519 INFO:teuthology.orchestra.run.smithi111.stderr:FINISHED --2024-04-24 22:18:00-- 2024-04-24T22:18:00.519 INFO:teuthology.orchestra.run.smithi111.stderr:Total wall clock time: 13s 2024-04-24T22:18:00.519 INFO:teuthology.orchestra.run.smithi111.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-24T22:18:00.522 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T22:18:00.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T22:18:02.766 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying... ######################################## 2024-04-24T22:18:02.921 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing... ######################################## 2024-04-24T22:18:03.044 INFO:teuthology.orchestra.run.smithi111.stdout:Updating / installing... 2024-04-24T22:18:11.004 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T22:18:50.816 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T22:18:50.816 DEBUG:teuthology.orchestra.run.smithi111:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T22:18:50.899 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T22:18:50.900 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T22:18:50.901 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T22:18:50.902 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T22:18:50.903 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T22:18:50.904 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T22:18:50.905 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T22:18:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T22:18:50.907 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T22:18:50.908 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T22:18:50.909 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T22:18:50.910 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T22:18:50.911 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T22:18:50.912 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T22:18:50.913 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T22:18:50.914 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T22:18:50.915 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T22:18:50.916 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T22:18:50.917 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T22:18:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T22:18:50.919 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T22:18:50.920 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T22:18:50.921 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T22:18:50.922 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T22:18:50.923 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T22:18:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T22:18:50.925 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T22:18:50.926 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T22:18:50.927 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T22:18:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T22:18:50.929 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T22:18:50.930 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T22:18:50.931 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T22:18:50.932 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T22:18:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T22:18:50.934 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T22:18:50.935 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T22:18:50.936 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T22:18:50.937 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T22:18:50.938 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T22:18:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T22:18:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T22:18:50.941 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T22:18:50.942 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T22:18:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T22:18:50.944 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T22:18:50.945 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T22:18:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T22:18:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T22:18:50.948 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T22:18:50.949 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T22:18:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T22:18:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T22:18:50.952 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T22:18:50.953 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T22:18:50.954 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T22:18:50.955 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T22:18:50.956 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T22:18:50.957 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T22:18:50.958 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T22:18:50.959 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T22:18:50.960 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T22:18:50.961 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T22:18:50.962 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T22:18:50.963 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T22:18:50.964 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T22:18:50.965 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T22:18:50.966 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T22:18:50.967 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T22:18:50.968 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T22:18:50.969 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T22:18:50.970 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T22:18:50.971 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T22:18:50.972 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T22:18:50.973 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T22:18:50.974 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T22:18:50.975 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T22:18:50.976 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T22:18:50.977 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T22:18:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T22:18:50.979 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T22:18:50.980 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T22:18:50.981 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T22:18:50.982 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T22:18:50.983 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T22:18:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T22:18:50.985 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T22:18:50.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T22:18:50.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T22:18:50.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T22:18:50.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T22:18:50.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T22:18:50.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T22:18:50.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T22:18:50.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T22:18:50.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T22:18:50.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T22:18:50.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T22:18:50.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T22:18:50.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T22:18:50.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T22:18:51.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T22:18:51.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T22:18:51.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T22:18:51.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T22:18:51.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T22:18:51.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T22:18:51.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T22:18:51.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T22:18:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T22:18:51.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T22:18:51.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T22:18:51.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T22:18:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T22:18:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T22:18:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T22:18:51.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T22:18:51.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T22:18:51.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T22:18:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T22:18:51.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T22:18:51.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T22:18:51.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T22:18:51.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T22:18:51.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T22:18:51.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T22:18:51.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T22:18:51.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T22:18:51.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T22:18:51.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T22:18:51.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T22:18:51.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T22:18:51.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T22:18:51.035 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:51.037 DEBUG:teuthology.orchestra.run.smithi111:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:for file in vmlinuz System.map config; do 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: fi 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:done 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T22:18:51.060 INFO:teuthology.orchestra.run.smithi111.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:51.061 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-24T22:18:51.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 68.el9 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-04-24T22:18:51.145 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8264254 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-04-24T22:18:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:18:51.147 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T22:18:51.148 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T22:18:51.148 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:51.148 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T22:18:51.696 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-04-24T22:18:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T22:18:53.132 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-04-24T22:18:53.134 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-04-24T22:18:53.149 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.zO4yKpPQHO 2024-04-24T22:18:53.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zO4yKpPQHO 2024-04-24T22:18:53.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.zO4yKpPQHO 2024-04-24T22:18:53.375 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.zO4yKpPQHO is 6KB 2024-04-24T22:18:53.394 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.zO4yKpPQHO 2024-04-24T22:18:53.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T22:18:53.469 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T22:18:53.469 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T22:18:53.469 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T22:18:53.469 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T22:18:53.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T22:18:53.607 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-04-24T22:19:23.610 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T22:19:23.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:19:23.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:19:41.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-24T22:19:50.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:19:50.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:19:54.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-24T22:20:06.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:20:06.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:20:09.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-24T22:20:24.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:20:24.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:20:42.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-24T22:21:00.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:21:00.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:21:01.033 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-24T22:21:01.660 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:21:01.661 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T22:21:01.661 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-04-24T22:21:01.700 INFO:teuthology.orchestra.run.smithi111.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T22:21:01.700 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T22:21:01.700 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T22:21:01.700 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T22:21:02.702 DEBUG:teuthology.parallel:result is None 2024-04-24T22:21:02.702 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:21:02.711 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:21:02.711 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:21:02.719 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:21:02.721 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:21:02.762 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:21:02.771 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:21:02.786 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:21:02.786 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:21:02.811 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:21:02.813 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:21:02.918 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:21:02.927 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:21:02.928 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:21:02.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:21:02.945 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:21:02.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:21:02.962 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:21:02.967 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:21:02.987 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:21:03.038 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.041 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.054 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.057 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.141 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.245 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:21:03.250 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:21:03.259 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:21:03.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:21:03.264 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:21:03.267 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:21:03.325 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:21:03.338 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:21:03.338 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:21:03.346 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:21:03.349 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:21:03.384 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:03.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:03.589 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:03.660 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:03.744 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:21:03.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:21:03.810 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:03.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:03.916 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:03.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:04.002 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:21:04.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:21:04.068 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:04.113 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:21:04.165 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:04.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:21:04.291 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:21:04.292 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:21:04.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-04-24T22:21:04.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-24T22:21:04.365 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-24T22:21:04.404 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:21:04.433 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:21:04.453 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:21:04.988 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:21:04.997 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:21:04.997 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:21:05.018 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:21:05.033 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:21:05.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-04-24T22:21:05.134 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-04-24T22:21:05.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-24T22:21:05.198 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-24T22:21:05.201 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-24T22:21:05.251 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-24T22:21:05.253 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:21:05.253 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-04-24T22:21:05.293 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-04-24T22:21:05.294 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-24T22:21:05.319 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-24T22:21:05.319 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-24T22:21:05.338 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-24T22:21:05.338 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:21:05.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:21:05.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:21:05.400 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:21:05.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:21:05.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:21:05.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:21:05.451 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:21:05.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:21:05.517 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:21:05.542 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:21:05.566 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:21:05.575 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T22:21:05.581 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T22:21:05.730 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:21:05.783 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:21:05.784 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi171.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:26:11.058 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-24T22:26:11.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:26:11.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:26:11.178 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-24T22:26:11.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-24T22:26:11.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-24T22:26:11.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:26:11.411 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-24T22:26:11.491 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-24T22:26:11.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-24T22:26:11.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:26:11.569 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-24T22:26:11.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-24T22:26:11.656 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:26:11.668 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:26:11.668 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:26:11.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:26:11.676 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:26:11.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:26:11.679 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:26:11.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:26:11.709 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:26:11.724 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:26:11.725 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:26:11.740 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:26:11.749 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-24T22:26:11.760 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-24T22:26:11.763 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:26:11.763 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-24T22:26:11.774 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:26:11.775 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-24T22:26:11.788 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:26:11.788 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:26:11.801 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:26:11.803 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:26:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.863 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.867 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-04-24T22:26:11.914 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-04-24T22:26:11.956 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:26:11.998 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:26:12.142 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:26:12.150 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:26:12.159 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:26:12.159 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:26:12.159 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:26:12.160 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:26:12.201 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:26:12.201 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T22:26:12.201 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:26:12.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:26:12.203 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:26:12.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:26:12.205 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:26:12.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:26:12.346 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:26:12.346 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:26:12.353 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:26:12.353 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:26:12.432 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:26:12.433 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:26:12.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:26:12.467 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:26:12.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:26:12.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:26:12.524 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:26:12.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:26:12.542 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:26:12.542 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:26:12.542 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:26:12.550 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:26:12.550 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:26:12.555 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:26:12.556 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:26:12.569 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:26:12.569 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:26:12.616 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:26:12.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:26:12.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:26:12.690 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:26:12.713 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:26:12.754 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-24T22:26:12.756 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-24T22:26:12.778 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-24T22:26:12.779 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-24T22:26:12.793 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:26:12.872 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-04-24T22:26:12.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-04-24T22:26:12.982 INFO:teuthology.orchestra.run.smithi186.stdout:48 files removed 2024-04-24T22:26:13.005 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-24T22:26:13.008 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:26:13.032 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:26:13.193 INFO:teuthology.orchestra.run.smithi111.stdout:48 files removed 2024-04-24T22:26:13.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:26:13.635 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 242 kB/s | 83 kB 00:00 2024-04-24T22:26:13.683 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-24T22:26:13.986 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-04-24T22:26:14.110 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-24T22:26:14.126 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-24T22:26:14.353 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-24T22:26:14.476 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-24T22:26:14.660 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-04-24T22:26:15.043 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-24T22:26:15.153 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-24T22:26:15.186 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-24T22:26:18.486 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-04-24T22:26:22.008 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-04-24T22:26:22.703 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T22:26:24.479 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-24T22:26:26.230 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:10 2024-04-24T22:26:26.480 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:26:30.000 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 19 MB 00:06 2024-04-24T22:26:30.904 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-24T22:26:31.563 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-24T22:26:32.575 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-24T22:26:33.698 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:26:33.698 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:26:33.710 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:26:33.710 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:26:33.711 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:26:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:26:33.713 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:26:33.716 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:26:33.717 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:26:33.778 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-24T22:26:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-24T22:26:33.797 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-24T22:26:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 889 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:26:33.798 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 221 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:26:33.799 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 746 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:26:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 298 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 170 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:26:33.801 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:26:33.802 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:26:33.803 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:26:33.804 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:26:33.805 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:26:33.805 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:26:33.805 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:26:33.805 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:26:33.805 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:26:33.806 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:26:33.807 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:26:33.808 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:26:33.809 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:26:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:26:33.811 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 4 Packages 2024-04-24T22:26:33.812 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:26:33.813 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 220 M 2024-04-24T22:26:33.813 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-24T22:26:34.542 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:26:35.273 INFO:teuthology.orchestra.run.smithi186.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 10 kB/s | 6.6 kB 00:00 2024-04-24T22:26:35.759 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-24T22:26:36.148 INFO:teuthology.orchestra.run.smithi186.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-24T22:26:36.424 INFO:teuthology.orchestra.run.smithi186.stdout:(3/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 12 MB/s | 21 MB 00:01 2024-04-24T22:26:36.449 INFO:teuthology.orchestra.run.smithi186.stdout:(4/152): ceph-immutable-object-cache-19.0.0-244 483 kB/s | 145 kB 00:00 2024-04-24T22:26:36.599 INFO:teuthology.orchestra.run.smithi186.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 2.4 MB 00:00 2024-04-24T22:26:37.065 INFO:teuthology.orchestra.run.smithi186.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-24T22:26:37.149 INFO:teuthology.orchestra.run.smithi186.stdout:(7/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-24T22:26:37.341 INFO:teuthology.orchestra.run.smithi186.stdout:(8/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-24T22:26:37.636 INFO:teuthology.orchestra.run.smithi186.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 85 kB/s | 25 kB 00:00 2024-04-24T22:26:37.776 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-24T22:26:38.033 INFO:teuthology.orchestra.run.smithi186.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 17 MB/s | 17 MB 00:00 2024-04-24T22:26:38.117 INFO:teuthology.orchestra.run.smithi186.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 221 kB 00:00 2024-04-24T22:26:38.200 INFO:teuthology.orchestra.run.smithi186.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 325 kB/s | 27 kB 00:00 2024-04-24T22:26:38.251 INFO:teuthology.orchestra.run.smithi186.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 633 kB/s | 32 kB 00:00 2024-04-24T22:26:38.359 INFO:teuthology.orchestra.run.smithi186.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 6.7 MB/s | 746 kB 00:00 2024-04-24T22:26:38.443 INFO:teuthology.orchestra.run.smithi186.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.9 MB/s | 163 kB 00:00 2024-04-24T22:26:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:(16/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 7.0 MB/s | 10 MB 00:01 2024-04-24T22:26:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 688 kB/s | 127 kB 00:00 2024-04-24T22:26:38.735 INFO:teuthology.orchestra.run.smithi186.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.7 MB/s | 501 kB 00:00 2024-04-24T22:26:38.894 INFO:teuthology.orchestra.run.smithi186.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 284 kB/s | 45 kB 00:00 2024-04-24T22:26:38.961 INFO:teuthology.orchestra.run.smithi186.stdout:(20/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 16 MB/s | 5.3 MB 00:00 2024-04-24T22:26:39.012 INFO:teuthology.orchestra.run.smithi186.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 3.2 MB/s | 162 kB 00:00 2024-04-24T22:26:39.045 INFO:teuthology.orchestra.run.smithi186.stdout:(22/152): python3-ceph-common-19.0.0-2445.gd2c4 883 kB/s | 133 kB 00:00 2024-04-24T22:26:39.129 INFO:teuthology.orchestra.run.smithi186.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 321 kB 00:00 2024-04-24T22:26:39.213 INFO:teuthology.orchestra.run.smithi186.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T22:26:39.238 INFO:teuthology.orchestra.run.smithi186.stdout:(25/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 298 kB 00:00 2024-04-24T22:26:39.263 INFO:teuthology.orchestra.run.smithi186.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-24T22:26:39.364 INFO:teuthology.orchestra.run.smithi186.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-24T22:26:39.422 INFO:teuthology.orchestra.run.smithi186.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 546 kB/s | 32 kB 00:00 2024-04-24T22:26:39.491 INFO:teuthology.orchestra.run.smithi186.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.0 MB/s | 141 kB 00:00 2024-04-24T22:26:39.656 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T22:26:39.683 INFO:teuthology.orchestra.run.smithi186.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 6.9 MB/s | 3.1 MB 00:00 2024-04-24T22:26:39.808 INFO:teuthology.orchestra.run.smithi186.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 11 MB/s | 3.5 MB 00:00 2024-04-24T22:26:40.209 INFO:teuthology.orchestra.run.smithi186.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 622 kB/s | 249 kB 00:00 2024-04-24T22:26:40.259 INFO:teuthology.orchestra.run.smithi186.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 994 kB/s | 49 kB 00:00 2024-04-24T22:26:40.317 INFO:teuthology.orchestra.run.smithi186.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 263 kB/s | 15 kB 00:00 2024-04-24T22:26:40.401 INFO:teuthology.orchestra.run.smithi186.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-24T22:26:40.493 INFO:teuthology.orchestra.run.smithi186.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-24T22:26:40.568 INFO:teuthology.orchestra.run.smithi186.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 244 kB/s | 18 kB 00:00 2024-04-24T22:26:40.885 INFO:teuthology.orchestra.run.smithi186.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-04-24T22:26:40.927 INFO:teuthology.orchestra.run.smithi186.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 224 kB/s | 80 kB 00:00 2024-04-24T22:26:41.186 INFO:teuthology.orchestra.run.smithi186.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 756 kB/s | 195 kB 00:00 2024-04-24T22:26:41.294 INFO:teuthology.orchestra.run.smithi186.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-24T22:26:41.345 INFO:teuthology.orchestra.run.smithi186.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-24T22:26:41.420 INFO:teuthology.orchestra.run.smithi186.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-24T22:26:41.487 INFO:teuthology.orchestra.run.smithi186.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-24T22:26:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-24T22:26:41.629 INFO:teuthology.orchestra.run.smithi186.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-24T22:26:41.704 INFO:teuthology.orchestra.run.smithi186.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-04-24T22:26:41.782 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:26:41.782 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:26:41.794 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:26:41.794 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:26:41.795 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:26:41.796 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:26:41.798 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:26:41.800 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:26:41.801 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:26:41.864 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 889 k 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:26:41.879 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 221 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:26:41.880 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 746 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 298 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:26:41.881 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 170 k 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:26:41.882 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:26:41.883 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:26:41.884 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:26:41.885 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:26:41.886 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:26:41.887 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:26:41.888 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:26:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-24T22:26:41.890 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:26:41.891 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 220 M 2024-04-24T22:26:41.891 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-24T22:26:42.080 INFO:teuthology.orchestra.run.smithi186.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-24T22:26:42.263 INFO:teuthology.orchestra.run.smithi186.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 943 kB 00:00 2024-04-24T22:26:42.330 INFO:teuthology.orchestra.run.smithi186.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 427 kB/s | 107 kB 00:00 2024-04-24T22:26:42.397 INFO:teuthology.orchestra.run.smithi186.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-04-24T22:26:42.547 INFO:teuthology.orchestra.run.smithi186.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 218 kB/s | 33 kB 00:00 2024-04-24T22:26:42.622 INFO:teuthology.orchestra.run.smithi186.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-04-24T22:26:42.756 INFO:teuthology.orchestra.run.smithi186.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-04-24T22:26:42.865 INFO:teuthology.orchestra.run.smithi186.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-24T22:26:43.082 INFO:teuthology.orchestra.run.smithi186.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 740 kB/s | 160 kB 00:00 2024-04-24T22:26:43.182 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-24T22:26:43.257 INFO:teuthology.orchestra.run.smithi186.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-04-24T22:26:43.458 INFO:teuthology.orchestra.run.smithi186.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-04-24T22:26:43.658 INFO:teuthology.orchestra.run.smithi186.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-24T22:26:43.665 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 7.4 MB/s | 5.4 MB 00:00 2024-04-24T22:26:43.732 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.1 MB/s | 145 kB 00:00 2024-04-24T22:26:43.808 INFO:teuthology.orchestra.run.smithi186.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 170 kB/s | 25 kB 00:00 2024-04-24T22:26:43.833 INFO:teuthology.orchestra.run.smithi186.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 1.5 MB/s | 582 kB 00:00 2024-04-24T22:26:43.984 INFO:teuthology.orchestra.run.smithi186.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 271 kB/s | 47 kB 00:00 2024-04-24T22:26:44.049 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-04-24T22:26:44.074 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 997 kB/s | 889 kB 00:00 2024-04-24T22:26:44.142 INFO:teuthology.orchestra.run.smithi186.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 948 kB/s | 292 kB 00:00 2024-04-24T22:26:44.233 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 8.1 MB/s | 1.5 MB 00:00 2024-04-24T22:26:44.276 INFO:teuthology.orchestra.run.smithi186.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 645 kB/s | 188 kB 00:00 2024-04-24T22:26:44.301 INFO:teuthology.orchestra.run.smithi186.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 222 kB/s | 35 kB 00:00 2024-04-24T22:26:44.875 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 11 MB/s | 21 MB 00:01 2024-04-24T22:26:45.251 INFO:teuthology.orchestra.run.smithi186.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 4.9 MB/s | 4.7 MB 00:00 2024-04-24T22:26:45.427 INFO:teuthology.orchestra.run.smithi186.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-24T22:26:45.801 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 17 MB 00:01 2024-04-24T22:26:45.852 INFO:teuthology.orchestra.run.smithi186.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 2.1 MB/s | 911 kB 00:00 2024-04-24T22:26:45.918 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 9.8 MB/s | 10 MB 00:01 2024-04-24T22:26:45.927 INFO:teuthology.orchestra.run.smithi186.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-04-24T22:26:45.934 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 189 kB/s | 25 kB 00:00 2024-04-24T22:26:46.001 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x8 2.4 MB/s | 4.7 MB 00:01 2024-04-24T22:26:46.035 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 221 kB 00:00 2024-04-24T22:26:46.061 INFO:teuthology.orchestra.run.smithi186.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 639 kB/s | 133 kB 00:00 2024-04-24T22:26:46.077 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 360 kB/s | 27 kB 00:00 2024-04-24T22:26:46.094 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 541 kB/s | 32 kB 00:00 2024-04-24T22:26:46.178 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.9 MB/s | 163 kB 00:00 2024-04-24T22:26:46.253 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-24T22:26:46.279 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 746 kB 00:00 2024-04-24T22:26:46.345 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.3 MB/s | 501 kB 00:00 2024-04-24T22:26:46.412 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 676 kB/s | 45 kB 00:00 2024-04-24T22:26:46.469 INFO:teuthology.orchestra.run.smithi186.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-04-24T22:26:46.487 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-24T22:26:46.563 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T22:26:46.646 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.8 MB/s | 321 kB 00:00 2024-04-24T22:26:46.730 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 298 kB 00:00 2024-04-24T22:26:46.805 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T22:26:46.881 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T22:26:46.945 INFO:teuthology.orchestra.run.smithi186.stdout:(72/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 5.2 MB/s | 48 MB 00:09 2024-04-24T22:26:47.011 INFO:teuthology.orchestra.run.smithi186.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 161 kB/s | 87 kB 00:00 2024-04-24T22:26:47.231 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.6 MB/s | 5.3 MB 00:00 2024-04-24T22:26:47.270 INFO:teuthology.orchestra.run.smithi186.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-04-24T22:26:47.289 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.6 MB/s | 3.1 MB 00:00 2024-04-24T22:26:47.315 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-24T22:26:47.337 INFO:teuthology.orchestra.run.smithi186.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 751 kB/s | 244 kB 00:00 2024-04-24T22:26:47.348 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 543 kB/s | 32 kB 00:00 2024-04-24T22:26:47.435 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.2 MB/s | 141 kB 00:00 2024-04-24T22:26:47.512 INFO:teuthology.orchestra.run.smithi186.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 146 kB/s | 26 kB 00:00 2024-04-24T22:26:47.579 INFO:teuthology.orchestra.run.smithi186.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 803 kB/s | 248 kB 00:00 2024-04-24T22:26:47.696 INFO:teuthology.orchestra.run.smithi186.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 104 kB/s | 19 kB 00:00 2024-04-24T22:26:47.732 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.2 MB/s | 3.5 MB 00:00 2024-04-24T22:26:47.791 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.2 MB/s | 249 kB 00:00 2024-04-24T22:26:47.830 INFO:teuthology.orchestra.run.smithi186.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 689 kB/s | 172 kB 00:00 2024-04-24T22:26:47.847 INFO:teuthology.orchestra.run.smithi186.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 231 kB/s | 35 kB 00:00 2024-04-24T22:26:47.858 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 743 kB/s | 49 kB 00:00 2024-04-24T22:26:47.917 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 261 kB/s | 15 kB 00:00 2024-04-24T22:26:47.984 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 264 kB 00:00 2024-04-24T22:26:48.092 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T22:26:48.147 INFO:teuthology.orchestra.run.smithi186.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-04-24T22:26:48.151 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 313 kB/s | 18 kB 00:00 2024-04-24T22:26:48.172 INFO:teuthology.orchestra.run.smithi186.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 770 kB/s | 250 kB 00:00 2024-04-24T22:26:48.268 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 690 kB/s | 80 kB 00:00 2024-04-24T22:26:48.385 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-24T22:26:48.389 INFO:teuthology.orchestra.run.smithi186.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 356 kB/s | 77 kB 00:00 2024-04-24T22:26:48.414 INFO:teuthology.orchestra.run.smithi186.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 832 kB/s | 222 kB 00:00 2024-04-24T22:26:48.552 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-04-24T22:26:48.556 INFO:teuthology.orchestra.run.smithi186.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 253 kB/s | 42 kB 00:00 2024-04-24T22:26:48.577 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T22:26:48.653 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-24T22:26:48.698 INFO:teuthology.orchestra.run.smithi186.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 949 kB/s | 269 kB 00:00 2024-04-24T22:26:48.703 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T22:26:48.815 INFO:teuthology.orchestra.run.smithi186.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 613 kB/s | 159 kB 00:00 2024-04-24T22:26:48.882 INFO:teuthology.orchestra.run.smithi186.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-04-24T22:26:48.966 INFO:teuthology.orchestra.run.smithi186.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.0 MB/s | 279 kB 00:00 2024-04-24T22:26:49.050 INFO:teuthology.orchestra.run.smithi186.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 218 kB/s | 51 kB 00:00 2024-04-24T22:26:49.083 INFO:teuthology.orchestra.run.smithi186.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 268 kB/s | 54 kB 00:00 2024-04-24T22:26:49.120 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 15 MB/s | 48 MB 00:03 2024-04-24T22:26:49.145 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 427 kB/s | 253 kB 00:00 2024-04-24T22:26:49.171 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 289 kB/s | 135 kB 00:00 2024-04-24T22:26:49.229 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-24T22:26:49.233 INFO:teuthology.orchestra.run.smithi186.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 227 kB/s | 42 kB 00:00 2024-04-24T22:26:49.254 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 934 kB/s | 125 kB 00:00 2024-04-24T22:26:49.259 INFO:teuthology.orchestra.run.smithi186.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 252 kB/s | 44 kB 00:00 2024-04-24T22:26:49.329 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-24T22:26:49.372 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 516 kB/s | 21 kB 00:00 2024-04-24T22:26:49.397 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 751 kB/s | 107 kB 00:00 2024-04-24T22:26:49.409 INFO:teuthology.orchestra.run.smithi186.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 240 kB/s | 36 kB 00:00 2024-04-24T22:26:49.422 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-24T22:26:49.447 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-24T22:26:49.484 INFO:teuthology.orchestra.run.smithi186.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 881 kB/s | 221 kB 00:00 2024-04-24T22:26:49.489 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-04-24T22:26:49.514 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-24T22:26:49.540 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-24T22:26:49.565 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T22:26:49.585 INFO:teuthology.orchestra.run.smithi186.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 270 kB/s | 47 kB 00:00 2024-04-24T22:26:49.590 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-24T22:26:49.624 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T22:26:49.682 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-24T22:26:49.708 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-24T22:26:49.733 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-24T22:26:49.735 INFO:teuthology.orchestra.run.smithi186.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-24T22:26:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-24T22:26:49.784 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-24T22:26:49.802 INFO:teuthology.orchestra.run.smithi186.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-04-24T22:26:49.809 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-24T22:26:49.834 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-24T22:26:49.926 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-24T22:26:49.935 INFO:teuthology.orchestra.run.smithi186.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 468 kB/s | 94 kB 00:00 2024-04-24T22:26:49.968 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-24T22:26:50.002 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T22:26:50.019 INFO:teuthology.orchestra.run.smithi186.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 294 kB/s | 64 kB 00:00 2024-04-24T22:26:50.027 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T22:26:50.086 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-24T22:26:50.094 INFO:teuthology.orchestra.run.smithi186.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-24T22:26:50.128 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-24T22:26:50.136 INFO:teuthology.orchestra.run.smithi186.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 223 kB/s | 45 kB 00:00 2024-04-24T22:26:50.153 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T22:26:50.195 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-24T22:26:50.262 INFO:teuthology.orchestra.run.smithi186.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-04-24T22:26:50.270 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-24T22:26:50.295 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-24T22:26:50.312 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-04-24T22:26:50.329 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 572 kB/s | 19 kB 00:00 2024-04-24T22:26:50.355 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-24T22:26:50.371 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-04-24T22:26:50.455 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-24T22:26:50.480 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-24T22:26:50.506 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T22:26:50.531 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T22:26:50.556 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-24T22:26:50.581 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T22:26:50.607 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T22:26:50.632 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-24T22:26:50.657 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T22:26:50.682 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T22:26:50.695 INFO:teuthology.orchestra.run.smithi186.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 981 kB/s | 548 kB 00:00 2024-04-24T22:26:50.724 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-24T22:26:50.745 INFO:teuthology.orchestra.run.smithi186.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 640 kB/s | 309 kB 00:00 2024-04-24T22:26:50.771 INFO:teuthology.orchestra.run.smithi186.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-04-24T22:26:50.791 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T22:26:50.817 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 480 kB/s | 44 kB 00:00 2024-04-24T22:26:50.842 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T22:26:50.854 INFO:teuthology.orchestra.run.smithi186.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-24T22:26:50.867 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-24T22:26:50.893 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-24T22:26:50.918 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T22:26:50.938 INFO:teuthology.orchestra.run.smithi186.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-04-24T22:26:50.943 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T22:26:50.969 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T22:26:50.994 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T22:26:51.005 INFO:teuthology.orchestra.run.smithi186.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-24T22:26:51.019 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-24T22:26:51.105 INFO:teuthology.orchestra.run.smithi186.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-24T22:26:51.253 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-24T22:26:51.278 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-24T22:26:51.313 INFO:teuthology.orchestra.run.smithi186.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-24T22:26:51.462 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-24T22:26:51.512 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-24T22:26:51.562 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-24T22:26:51.581 INFO:teuthology.orchestra.run.smithi186.stdout:(112/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-04-24T22:26:51.587 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-24T22:26:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-04-24T22:26:51.621 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-04-24T22:26:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-24T22:26:51.704 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 810 kB/s | 67 kB 00:00 2024-04-24T22:26:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-04-24T22:26:51.730 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 341 kB/s | 49 kB 00:00 2024-04-24T22:26:51.797 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-24T22:26:51.815 INFO:teuthology.orchestra.run.smithi186.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-04-24T22:26:51.822 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-24T22:26:51.841 INFO:teuthology.orchestra.run.smithi186.stdout:(117/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 167 kB/s | 43 kB 00:00 2024-04-24T22:26:51.855 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-24T22:26:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-24T22:26:51.898 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-04-24T22:26:51.915 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-04-24T22:26:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-24T22:26:51.940 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-24T22:26:51.957 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-04-24T22:26:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-24T22:26:51.982 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-24T22:26:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-24T22:26:52.007 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-24T22:26:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-04-24T22:26:52.033 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-24T22:26:52.049 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 525 kB/s | 35 kB 00:00 2024-04-24T22:26:52.075 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-24T22:26:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-24T22:26:52.100 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-24T22:26:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-24T22:26:52.126 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 211 kB/s | 11 kB 00:00 2024-04-24T22:26:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-24T22:26:52.151 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-24T22:26:52.168 INFO:teuthology.orchestra.run.smithi186.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-24T22:26:52.176 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-24T22:26:52.193 INFO:teuthology.orchestra.run.smithi186.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-24T22:26:52.218 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-24T22:26:52.218 INFO:teuthology.orchestra.run.smithi186.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-04-24T22:26:52.243 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-24T22:26:52.269 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-24T22:26:52.269 INFO:teuthology.orchestra.run.smithi186.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-04-24T22:26:52.294 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-04-24T22:26:52.336 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-24T22:26:52.353 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-04-24T22:26:52.377 INFO:teuthology.orchestra.run.smithi186.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-04-24T22:26:52.378 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-24T22:26:52.403 INFO:teuthology.orchestra.run.smithi186.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-04-24T22:26:52.403 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-24T22:26:52.428 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-24T22:26:52.436 INFO:teuthology.orchestra.run.smithi186.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-24T22:26:52.454 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T22:26:52.461 INFO:teuthology.orchestra.run.smithi186.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-24T22:26:52.479 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-24T22:26:52.504 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-24T22:26:52.545 INFO:teuthology.orchestra.run.smithi186.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-24T22:26:52.554 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-24T22:26:52.571 INFO:teuthology.orchestra.run.smithi186.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-24T22:26:52.580 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-24T22:26:52.595 INFO:teuthology.orchestra.run.smithi186.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 673 kB/s | 90 kB 00:00 2024-04-24T22:26:52.605 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-04-24T22:26:52.622 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-04-24T22:26:52.646 INFO:teuthology.orchestra.run.smithi186.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-24T22:26:52.647 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-24T22:26:52.673 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-24T22:26:52.688 INFO:teuthology.orchestra.run.smithi186.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-24T22:26:52.698 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-24T22:26:52.721 INFO:teuthology.orchestra.run.smithi186.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-24T22:26:52.723 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-24T22:26:52.747 INFO:teuthology.orchestra.run.smithi186.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-24T22:26:52.748 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T22:26:52.780 INFO:teuthology.orchestra.run.smithi186.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-24T22:26:52.790 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-24T22:26:52.806 INFO:teuthology.orchestra.run.smithi186.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-04-24T22:26:52.831 INFO:teuthology.orchestra.run.smithi186.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-24T22:26:52.856 INFO:teuthology.orchestra.run.smithi186.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T22:26:52.898 INFO:teuthology.orchestra.run.smithi186.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-24T22:26:52.924 INFO:teuthology.orchestra.run.smithi186.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-24T22:26:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-24T22:26:53.033 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 11 MB/s | 3.4 MB 00:00 2024-04-24T22:26:53.083 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-04-24T22:26:53.107 INFO:teuthology.orchestra.run.smithi186.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-24T22:26:53.274 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-04-24T22:26:53.474 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-04-24T22:26:53.478 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:26:53.478 INFO:teuthology.orchestra.run.smithi171.stdout:Total 19 MB/s | 220 MB 00:11 2024-04-24T22:26:53.508 INFO:teuthology.orchestra.run.smithi186.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-24T22:26:53.541 INFO:teuthology.orchestra.run.smithi186.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-24T22:26:53.675 INFO:teuthology.orchestra.run.smithi186.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.5 MB/s | 3.4 MB 00:00 2024-04-24T22:26:54.058 INFO:teuthology.orchestra.run.smithi186.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 2.4 MB 00:00 2024-04-24T22:26:54.065 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:26:54.065 INFO:teuthology.orchestra.run.smithi186.stdout:Total 11 MB/s | 220 MB 00:20 2024-04-24T22:26:54.582 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-24T22:26:54.725 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-24T22:26:54.725 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-24T22:26:55.191 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-24T22:26:55.338 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-24T22:26:55.338 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-24T22:26:56.493 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-24T22:26:56.493 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-24T22:26:57.118 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-24T22:26:57.119 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-24T22:26:57.734 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-24T22:27:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-24T22:27:02.553 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:27:02.600 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:27:02.655 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:27:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:27:02.721 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:27:02.764 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:27:03.014 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-24T22:27:03.106 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:27:03.142 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:27:03.640 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:27:03.642 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T22:27:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:27:03.708 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:27:03.736 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:03.766 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:03.840 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:27:03.887 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:27:03.916 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:27:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:27:04.007 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:04.037 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:27:04.147 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:27:04.179 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:27:04.186 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:27:04.233 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:27:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:04.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:27:04.356 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:04.379 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:27:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:27:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:27:04.441 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:27:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:27:04.463 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:27:04.484 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:27:04.529 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:27:04.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:27:04.615 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:27:04.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:27:04.675 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:27:04.680 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:27:04.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:27:04.745 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:27:04.798 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:27:04.826 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:27:04.861 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:27:04.914 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:27:04.918 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:27:04.963 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:27:04.991 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:27:04.997 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:27:05.025 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:27:05.068 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:27:05.080 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:27:05.096 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:27:05.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:27:05.166 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:27:05.183 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:27:05.220 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:27:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:27:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:27:05.294 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:27:05.321 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:27:05.374 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:27:05.403 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:27:05.429 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:27:05.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:27:05.480 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:27:05.516 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:27:05.558 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:27:05.610 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:27:05.638 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:27:05.708 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:27:05.753 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:27:05.828 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:27:05.963 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:27:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:27:06.014 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:27:06.069 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:27:06.070 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:27:06.091 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:27:06.092 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:27:06.093 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:27:06.095 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:27:06.097 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:27:06.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:27:06.101 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:27:06.102 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:27:06.130 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:27:06.172 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-24T22:27:06.172 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:27:06.196 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 889 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 221 k 2024-04-24T22:27:06.197 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 746 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:27:06.198 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 298 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 170 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:27:06.199 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:27:06.200 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:27:06.201 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:27:06.202 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:27:06.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:27:06.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:27:06.206 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:27:06.209 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:27:06.209 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:27:06.210 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:27:06.210 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:27:06.210 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:27:06.210 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:27:06.210 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:27:06.211 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:27:06.211 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:27:06.211 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:27:06.211 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:27:06.211 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-24T22:27:06.212 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-04-24T22:27:06.213 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 4 Packages 2024-04-24T22:27:06.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:27:06.214 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:27:06.214 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 220 M 2024-04-24T22:27:06.214 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-04-24T22:27:06.252 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:27:06.413 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:27:06.463 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:27:06.518 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:27:06.545 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:27:06.599 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:27:06.630 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:27:06.634 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:27:06.663 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:27:06.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:27:06.706 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:27:06.713 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:27:06.744 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:27:06.750 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:27:06.792 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:27:06.793 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:27:06.835 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:27:06.968 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:27:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:27:07.021 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:27:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:27:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:27:07.076 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:27:07.095 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:27:07.174 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:27:07.190 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:27:07.206 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:27:07.229 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:27:07.242 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:27:07.269 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:27:07.289 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:27:07.295 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:27:07.319 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:27:07.327 INFO:teuthology.orchestra.run.smithi111.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-24T22:27:07.333 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:27:07.377 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:27:07.394 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:27:07.418 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:27:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:27:07.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:27:07.560 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:27:07.588 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:27:07.598 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:27:07.622 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:27:07.629 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:27:07.662 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:27:07.670 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:27:07.693 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:27:07.723 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:27:07.757 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:27:07.764 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:27:07.791 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:27:07.804 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:27:07.841 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:27:07.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:27:07.875 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:27:07.878 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:27:07.903 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:27:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:27:07.985 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:27:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:27:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:27:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:27:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:27:08.143 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:27:08.171 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:27:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:27:08.238 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:27:08.238 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:27:08.252 INFO:teuthology.orchestra.run.smithi111.stdout:(2/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 4.3 MB/s | 5.4 MB 00:01 2024-04-24T22:27:08.319 INFO:teuthology.orchestra.run.smithi111.stdout:(3/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 898 kB/s | 889 kB 00:00 2024-04-24T22:27:08.353 INFO:teuthology.orchestra.run.smithi111.stdout:(4/152): ceph-immutable-object-cache-19.0.0-244 1.4 MB/s | 145 kB 00:00 2024-04-24T22:27:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:27:08.543 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:27:08.573 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:27:08.616 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:27:08.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:27:08.699 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:27:08.704 INFO:teuthology.orchestra.run.smithi111.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-04-24T22:27:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:27:08.823 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:27:08.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:27:08.903 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:27:08.950 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:27:08.959 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:27:09.002 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:27:09.036 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:27:09.048 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:27:09.105 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:27:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:27:09.185 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:27:09.275 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:27:09.321 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:27:09.500 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:27:09.527 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:27:09.606 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:27:09.613 INFO:teuthology.orchestra.run.smithi111.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-04-24T22:27:09.829 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:27:09.876 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:27:09.910 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:27:10.010 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:27:10.061 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:27:10.155 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:27:10.195 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:27:10.204 INFO:teuthology.orchestra.run.smithi111.stdout:(7/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-24T22:27:10.248 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:27:10.368 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:27:10.376 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:27:10.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:27:10.488 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:27:10.596 INFO:teuthology.orchestra.run.smithi111.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 5.8 MB/s | 21 MB 00:03 2024-04-24T22:27:10.646 INFO:teuthology.orchestra.run.smithi111.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 516 kB/s | 25 kB 00:00 2024-04-24T22:27:11.140 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:27:11.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:27:11.599 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:27:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:27:11.738 INFO:teuthology.orchestra.run.smithi111.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 7.9 MB/s | 17 MB 00:02 2024-04-24T22:27:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:27:11.805 INFO:teuthology.orchestra.run.smithi111.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 221 kB 00:00 2024-04-24T22:27:11.864 INFO:teuthology.orchestra.run.smithi111.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 472 kB/s | 27 kB 00:00 2024-04-24T22:27:11.922 INFO:teuthology.orchestra.run.smithi111.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 553 kB/s | 32 kB 00:00 2024-04-24T22:27:12.056 INFO:teuthology.orchestra.run.smithi111.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.5 MB/s | 746 kB 00:00 2024-04-24T22:27:12.062 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:27:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:27:12.148 INFO:teuthology.orchestra.run.smithi111.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.8 MB/s | 163 kB 00:00 2024-04-24T22:27:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:27:12.202 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:27:12.206 INFO:teuthology.orchestra.run.smithi111.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 2.2 MB/s | 127 kB 00:00 2024-04-24T22:27:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:27:12.315 INFO:teuthology.orchestra.run.smithi111.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.6 MB/s | 501 kB 00:00 2024-04-24T22:27:12.481 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:27:12.572 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:27:12.600 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:27:12.633 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:27:12.711 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:27:12.733 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:27:12.788 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:27:12.818 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:27:12.955 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:27:12.995 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:27:13.028 INFO:teuthology.orchestra.run.smithi111.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.5 MB/s | 5.3 MB 00:00 2024-04-24T22:27:13.086 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:27:13.107 INFO:teuthology.orchestra.run.smithi111.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 582 kB/s | 45 kB 00:00 2024-04-24T22:27:13.117 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:27:13.153 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:27:13.166 INFO:teuthology.orchestra.run.smithi111.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.3 MB/s | 133 kB 00:00 2024-04-24T22:27:13.241 INFO:teuthology.orchestra.run.smithi111.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T22:27:13.316 INFO:teuthology.orchestra.run.smithi111.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.2 MB/s | 321 kB 00:00 2024-04-24T22:27:13.392 INFO:teuthology.orchestra.run.smithi111.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 298 kB 00:00 2024-04-24T22:27:13.459 INFO:teuthology.orchestra.run.smithi111.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 99 kB 00:00 2024-04-24T22:27:13.517 INFO:teuthology.orchestra.run.smithi111.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-24T22:27:13.628 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:27:13.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:27:13.712 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:27:13.742 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:27:13.943 INFO:teuthology.orchestra.run.smithi111.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.3 MB/s | 3.1 MB 00:00 2024-04-24T22:27:14.010 INFO:teuthology.orchestra.run.smithi111.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-24T22:27:14.077 INFO:teuthology.orchestra.run.smithi111.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 483 kB/s | 32 kB 00:00 2024-04-24T22:27:14.135 INFO:teuthology.orchestra.run.smithi111.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-24T22:27:14.511 INFO:teuthology.orchestra.run.smithi111.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.4 MB/s | 3.5 MB 00:00 2024-04-24T22:27:15.058 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:15.312 INFO:teuthology.orchestra.run.smithi111.stdout:(31/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 10 MB/s | 48 MB 00:04 2024-04-24T22:27:15.470 INFO:teuthology.orchestra.run.smithi111.stdout:(32/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 1.9 MB/s | 10 MB 00:05 2024-04-24T22:27:15.503 INFO:teuthology.orchestra.run.smithi111.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.3 MB/s | 249 kB 00:00 2024-04-24T22:27:15.579 INFO:teuthology.orchestra.run.smithi111.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 206 kB/s | 15 kB 00:00 2024-04-24T22:27:15.629 INFO:teuthology.orchestra.run.smithi111.stdout:(35/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 312 kB/s | 49 kB 00:00 2024-04-24T22:27:15.652 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:15.662 INFO:teuthology.orchestra.run.smithi111.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-24T22:27:15.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:27:15.746 INFO:teuthology.orchestra.run.smithi111.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 221 kB/s | 18 kB 00:00 2024-04-24T22:27:15.776 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:27:15.776 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:27:15.776 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:27:15.776 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:15.803 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:27:15.889 INFO:teuthology.orchestra.run.smithi111.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-24T22:27:15.988 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:15.997 INFO:teuthology.orchestra.run.smithi111.stdout:(39/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 2.0 MB/s | 762 kB 00:00 2024-04-24T22:27:16.021 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:16.022 INFO:teuthology.orchestra.run.smithi111.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 293 kB/s | 80 kB 00:00 2024-04-24T22:27:16.122 INFO:teuthology.orchestra.run.smithi111.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-24T22:27:16.156 INFO:teuthology.orchestra.run.smithi111.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-24T22:27:16.206 INFO:teuthology.orchestra.run.smithi111.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-24T22:27:16.232 INFO:teuthology.orchestra.run.smithi111.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-24T22:27:16.265 INFO:teuthology.orchestra.run.smithi111.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T22:27:16.299 INFO:teuthology.orchestra.run.smithi111.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-24T22:27:16.349 INFO:teuthology.orchestra.run.smithi111.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-24T22:27:16.383 INFO:teuthology.orchestra.run.smithi111.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-24T22:27:16.566 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:27:16.608 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:27:16.698 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:27:16.699 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:27:16.699 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:27:16.699 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:16.726 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:27:16.816 INFO:teuthology.orchestra.run.smithi111.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-04-24T22:27:16.942 INFO:teuthology.orchestra.run.smithi111.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 173 kB/s | 21 kB 00:00 2024-04-24T22:27:17.075 INFO:teuthology.orchestra.run.smithi111.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 889 kB/s | 689 kB 00:00 2024-04-24T22:27:17.201 INFO:teuthology.orchestra.run.smithi111.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 33 kB 00:00 2024-04-24T22:27:17.251 INFO:teuthology.orchestra.run.smithi111.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-04-24T22:27:17.293 INFO:teuthology.orchestra.run.smithi111.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 896 kB/s | 313 kB 00:00 2024-04-24T22:27:17.376 INFO:teuthology.orchestra.run.smithi111.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 146 kB/s | 18 kB 00:00 2024-04-24T22:27:17.418 INFO:teuthology.orchestra.run.smithi111.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-24T22:27:17.519 INFO:teuthology.orchestra.run.smithi111.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-04-24T22:27:17.561 INFO:teuthology.orchestra.run.smithi111.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-04-24T22:27:17.728 INFO:teuthology.orchestra.run.smithi111.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-24T22:27:17.811 INFO:teuthology.orchestra.run.smithi111.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.9 MB/s | 3.0 MB 00:00 2024-04-24T22:27:17.853 INFO:teuthology.orchestra.run.smithi111.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-04-24T22:27:17.887 INFO:teuthology.orchestra.run.smithi111.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-04-24T22:27:17.937 INFO:teuthology.orchestra.run.smithi111.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 380 kB/s | 47 kB 00:00 2024-04-24T22:27:17.971 INFO:teuthology.orchestra.run.smithi111.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-24T22:27:18.013 INFO:teuthology.orchestra.run.smithi111.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-24T22:27:18.072 INFO:teuthology.orchestra.run.smithi111.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 264 kB/s | 35 kB 00:00 2024-04-24T22:27:18.230 INFO:teuthology.orchestra.run.smithi111.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-24T22:27:18.638 INFO:teuthology.orchestra.run.smithi111.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-24T22:27:18.739 INFO:teuthology.orchestra.run.smithi111.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.2 MB/s | 4.7 MB 00:00 2024-04-24T22:27:18.906 INFO:teuthology.orchestra.run.smithi111.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 501 kB/s | 133 kB 00:00 2024-04-24T22:27:19.498 INFO:teuthology.orchestra.run.smithi111.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 719 kB/s | 911 kB 00:01 2024-04-24T22:27:19.690 INFO:teuthology.orchestra.run.smithi111.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 457 kB/s | 87 kB 00:00 2024-04-24T22:27:20.299 INFO:teuthology.orchestra.run.smithi111.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 742 kB/s | 1.0 MB 00:01 2024-04-24T22:27:20.649 INFO:teuthology.orchestra.run.smithi111.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 699 kB/s | 244 kB 00:00 2024-04-24T22:27:21.008 INFO:teuthology.orchestra.run.smithi111.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 693 kB/s | 248 kB 00:00 2024-04-24T22:27:21.108 INFO:teuthology.orchestra.run.smithi111.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-24T22:27:21.208 INFO:teuthology.orchestra.run.smithi111.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-04-24T22:27:21.492 INFO:teuthology.orchestra.run.smithi111.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 609 kB/s | 172 kB 00:00 2024-04-24T22:27:21.592 INFO:teuthology.orchestra.run.smithi111.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 349 kB/s | 35 kB 00:00 2024-04-24T22:27:24.393 INFO:teuthology.orchestra.run.smithi111.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-04-24T22:27:24.751 INFO:teuthology.orchestra.run.smithi111.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 702 kB/s | 250 kB 00:00 2024-04-24T22:27:25.093 INFO:teuthology.orchestra.run.smithi111.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 651 kB/s | 222 kB 00:00 2024-04-24T22:27:25.285 INFO:teuthology.orchestra.run.smithi111.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-04-24T22:27:25.410 INFO:teuthology.orchestra.run.smithi111.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 340 kB/s | 42 kB 00:00 2024-04-24T22:27:25.844 INFO:teuthology.orchestra.run.smithi111.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 622 kB/s | 269 kB 00:00 2024-04-24T22:27:26.111 INFO:teuthology.orchestra.run.smithi111.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 597 kB/s | 159 kB 00:00 2024-04-24T22:27:26.536 INFO:teuthology.orchestra.run.smithi111.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 657 kB/s | 279 kB 00:00 2024-04-24T22:27:26.661 INFO:teuthology.orchestra.run.smithi111.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 410 kB/s | 51 kB 00:00 2024-04-24T22:27:26.787 INFO:teuthology.orchestra.run.smithi111.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 431 kB/s | 54 kB 00:00 2024-04-24T22:27:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:27:27.389 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-24T22:27:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:27:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:27:27.566 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:27.566 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:27:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:27.637 INFO:teuthology.orchestra.run.smithi111.stdout:(90/152): python3-babel-2.9.1-2.el9.noarch.rpm 768 kB/s | 6.0 MB 00:07 2024-04-24T22:27:27.762 INFO:teuthology.orchestra.run.smithi111.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 336 kB/s | 42 kB 00:00 2024-04-24T22:27:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 355 kB/s | 44 kB 00:00 2024-04-24T22:27:28.089 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-24T22:27:28.090 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:28.252 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:27:28.268 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:27:28.268 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:28.268 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:27:28.268 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:28.270 INFO:teuthology.orchestra.run.smithi111.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 579 kB/s | 221 kB 00:00 2024-04-24T22:27:28.396 INFO:teuthology.orchestra.run.smithi111.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 287 kB/s | 36 kB 00:00 2024-04-24T22:27:28.505 INFO:teuthology.orchestra.run.smithi111.stdout:(95/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 755 kB/s | 5.1 MB 00:06 2024-04-24T22:27:28.547 INFO:teuthology.orchestra.run.smithi111.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 316 kB/s | 47 kB 00:00 2024-04-24T22:27:28.697 INFO:teuthology.orchestra.run.smithi111.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 387 kB/s | 58 kB 00:00 2024-04-24T22:27:28.897 INFO:teuthology.orchestra.run.smithi111.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 470 kB/s | 94 kB 00:00 2024-04-24T22:27:28.972 INFO:teuthology.orchestra.run.smithi111.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 653 kB/s | 305 kB 00:00 2024-04-24T22:27:29.036 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:27:29.062 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:27:29.063 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:29.063 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:27:29.063 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:27:29.063 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:27:29.063 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:29.098 INFO:teuthology.orchestra.run.smithi111.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-04-24T22:27:29.140 INFO:teuthology.orchestra.run.smithi111.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 264 kB/s | 64 kB 00:00 2024-04-24T22:27:29.282 INFO:teuthology.orchestra.run.smithi111.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-04-24T22:27:29.286 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:27:29.309 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:27:29.309 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:29.309 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:27:29.310 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:27:29.310 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:27:29.310 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:29.474 INFO:teuthology.orchestra.run.smithi111.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-04-24T22:27:29.541 INFO:teuthology.orchestra.run.smithi111.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-04-24T22:27:29.608 INFO:teuthology.orchestra.run.smithi111.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 294 kB/s | 19 kB 00:00 2024-04-24T22:27:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:27:29.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:27:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:27:29.776 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:29.777 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:27:29.777 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:27:29.777 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:27:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:29.791 INFO:teuthology.orchestra.run.smithi111.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:27:29.798 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:27:29.908 INFO:teuthology.orchestra.run.smithi111.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-24T22:27:29.958 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:27:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:27:30.032 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:27:30.042 INFO:teuthology.orchestra.run.smithi111.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-24T22:27:30.055 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:27:30.055 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:30.055 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:27:30.055 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:27:30.055 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:27:30.056 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:30.058 INFO:teuthology.orchestra.run.smithi111.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-04-24T22:27:30.117 INFO:teuthology.orchestra.run.smithi111.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 905 kB/s | 67 kB 00:00 2024-04-24T22:27:30.160 INFO:teuthology.orchestra.run.smithi111.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-24T22:27:30.208 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:27:30.226 INFO:teuthology.orchestra.run.smithi111.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-24T22:27:30.254 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:27:30.268 INFO:teuthology.orchestra.run.smithi111.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-24T22:27:30.294 INFO:teuthology.orchestra.run.smithi111.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 657 kB/s | 43 kB 00:00 2024-04-24T22:27:30.319 INFO:teuthology.orchestra.run.smithi111.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 648 kB/s | 32 kB 00:00 2024-04-24T22:27:30.334 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:27:30.336 INFO:teuthology.orchestra.run.smithi111.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 364 kB/s | 15 kB 00:00 2024-04-24T22:27:30.370 INFO:teuthology.orchestra.run.smithi111.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-24T22:27:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:27:30.461 INFO:teuthology.orchestra.run.smithi111.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-24T22:27:30.495 INFO:teuthology.orchestra.run.smithi111.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-24T22:27:30.499 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:27:30.520 INFO:teuthology.orchestra.run.smithi111.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:27:30.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:30.554 INFO:teuthology.orchestra.run.smithi111.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-24T22:27:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:27:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:27:30.613 INFO:teuthology.orchestra.run.smithi111.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-24T22:27:30.630 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:27:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:27:30.655 INFO:teuthology.orchestra.run.smithi111.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 262 kB/s | 11 kB 00:00 2024-04-24T22:27:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:27:30.686 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:27:30.739 INFO:teuthology.orchestra.run.smithi111.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-24T22:27:30.741 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:27:30.763 INFO:teuthology.orchestra.run.smithi111.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-24T22:27:30.790 INFO:teuthology.orchestra.run.smithi111.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-04-24T22:27:30.814 INFO:teuthology.orchestra.run.smithi111.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-04-24T22:27:30.839 INFO:teuthology.orchestra.run.smithi111.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 401 kB/s | 20 kB 00:00 2024-04-24T22:27:30.890 INFO:teuthology.orchestra.run.smithi111.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 945 kB/s | 46 kB 00:00 2024-04-24T22:27:30.935 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:27:30.940 INFO:teuthology.orchestra.run.smithi111.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-24T22:27:30.965 INFO:teuthology.orchestra.run.smithi111.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T22:27:30.992 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:27:31.016 INFO:teuthology.orchestra.run.smithi111.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 776 kB/s | 58 kB 00:00 2024-04-24T22:27:31.049 INFO:teuthology.orchestra.run.smithi111.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-24T22:27:31.068 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:27:31.074 INFO:teuthology.orchestra.run.smithi111.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-04-24T22:27:31.100 INFO:teuthology.orchestra.run.smithi111.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-24T22:27:31.113 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:27:31.125 INFO:teuthology.orchestra.run.smithi111.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 571 kB/s | 28 kB 00:00 2024-04-24T22:27:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:27:31.150 INFO:teuthology.orchestra.run.smithi111.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 623 kB/s | 31 kB 00:00 2024-04-24T22:27:31.157 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:27:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:27:31.184 INFO:teuthology.orchestra.run.smithi111.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-24T22:27:31.210 INFO:teuthology.orchestra.run.smithi111.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-24T22:27:31.235 INFO:teuthology.orchestra.run.smithi111.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 721 kB/s | 36 kB 00:00 2024-04-24T22:27:31.240 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:27:31.260 INFO:teuthology.orchestra.run.smithi111.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-24T22:27:31.269 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:27:31.293 INFO:teuthology.orchestra.run.smithi111.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-24T22:27:31.295 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:27:31.319 INFO:teuthology.orchestra.run.smithi111.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-24T22:27:31.333 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:27:31.352 INFO:teuthology.orchestra.run.smithi111.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-24T22:27:31.365 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:27:31.378 INFO:teuthology.orchestra.run.smithi111.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-04-24T22:27:31.393 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:27:31.411 INFO:teuthology.orchestra.run.smithi111.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T22:27:31.503 INFO:teuthology.orchestra.run.smithi111.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T22:27:31.829 INFO:teuthology.orchestra.run.smithi111.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-04-24T22:27:31.842 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:27:31.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:27:31.906 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:27:31.977 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:27:32.006 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:27:32.071 INFO:teuthology.orchestra.run.smithi111.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-04-24T22:27:32.204 INFO:teuthology.orchestra.run.smithi111.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-24T22:27:32.303 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:27:32.337 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:27:32.446 INFO:teuthology.orchestra.run.smithi111.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-24T22:27:33.043 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:27:33.083 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:27:33.304 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:27:33.329 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:27:33.433 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:27:33.601 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:27:33.631 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:27:33.655 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:27:33.655 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:33.656 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:27:33.656 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:27:33.656 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:27:33.656 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:33.707 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:27:33.719 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:27:33.816 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:27:33.872 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:27:34.052 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:27:34.083 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:27:34.185 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:27:34.347 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:27:34.377 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:27:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:27:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:27:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:27:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:27:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:34.452 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:27:34.464 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:27:34.561 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:27:34.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:27:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:27:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:27:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:27:34.839 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:34.839 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:27:34.840 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:27:34.840 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:27:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:27:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:27:34.883 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:34.883 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:27:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:34.928 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:27:34.950 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:27:34.950 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:34.951 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:27:34.951 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:27:34.951 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:27:34.951 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:27:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:27:35.285 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:35.286 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:27:35.286 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:27:35.286 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:27:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:35.511 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:27:35.538 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:27:35.561 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:27:35.562 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:35.562 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:27:35.562 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:27:35.562 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:27:35.562 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:35.605 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:27:35.627 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:27:35.628 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:35.628 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:27:35.628 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:35.682 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:27:35.704 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:27:35.704 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:35.704 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:27:35.704 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:27:35.704 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:27:35.705 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:36.007 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:27:36.030 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:27:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:27:39.881 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:27:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:27:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:27:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:27:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:27:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:27:40.552 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:27:40.594 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:27:40.634 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:27:40.667 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:27:40.693 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:27:40.808 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:40.848 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:40.869 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:40.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:27:40.915 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:27:40.945 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:27:41.516 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:41.558 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:41.578 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:27:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:27:41.796 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:27:41.837 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:27:41.918 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:27:41.950 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:27:41.950 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:27:42.017 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:27:42.058 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:27:42.059 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:27:42.151 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:27:42.215 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:42.413 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:27:42.496 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:27:42.537 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:27:42.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:27:42.658 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:27:42.659 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:27:42.743 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:27:42.775 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:27:42.775 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:27:42.867 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:27:42.931 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:27:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:27:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:27:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:27:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:27:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:27:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:27:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:27:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:27:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:27:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:27:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:27:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:27:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:27:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:27:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:27:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:27:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:27:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:27:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:27:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:27:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:27:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:27:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:27:48.828 INFO:teuthology.orchestra.run.smithi111.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 755 kB/s | 16 MB 00:22 2024-04-24T22:27:48.834 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:27:48.835 INFO:teuthology.orchestra.run.smithi111.stdout:Total 5.2 MB/s | 220 MB 00:42 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:27:48.859 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:27:48.860 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:27:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:27:48.862 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:27:48.862 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:27:48.862 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:27:48.878 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:27:48.878 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:27:48.879 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:27:48.880 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:27:48.881 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:27:48.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:27:48.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:27:48.884 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:27:48.885 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:27:48.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:27:48.887 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:27:48.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:27:48.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:27:48.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:27:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:27:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:27:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:27:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:27:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:27:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:27:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:27:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:27:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:27:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:27:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:27:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:27:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:27:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-24T22:27:50.458 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-24T22:27:50.834 DEBUG:teuthology.parallel:result is None 2024-04-24T22:27:50.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:27:50.969 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.970 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.971 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:27:50.972 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.973 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:27:50.974 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:27:50.976 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:27:50.977 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.978 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:27:50.979 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:27:50.980 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:27:50.981 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:27:50.981 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:27:50.981 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-24T22:27:51.370 DEBUG:teuthology.parallel:result is None 2024-04-24T22:27:51.743 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-04-24T22:27:51.982 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-04-24T22:27:51.982 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-04-24T22:27:54.443 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-04-24T22:27:54.444 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-04-24T22:28:00.785 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-04-24T22:28:00.885 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:28:00.970 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:28:01.079 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:28:01.139 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:28:01.174 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:28:01.321 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:28:01.745 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:28:01.844 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:28:02.032 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:28:02.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:28:02.225 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:28:02.278 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:28:02.616 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:28:02.683 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:28:02.708 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:28:02.918 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:28:02.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:28:03.038 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:28:03.068 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:28:03.148 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:28:03.616 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:28:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:28:03.876 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:28:04.002 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:28:04.052 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:28:04.101 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:28:04.234 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:28:04.282 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:28:04.345 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:28:04.394 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:28:04.434 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:28:04.552 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:28:04.620 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:28:04.652 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:28:04.709 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:28:04.815 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:28:04.855 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:28:04.904 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:28:04.937 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:28:05.052 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:28:05.115 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:28:05.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:28:05.391 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:28:05.460 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:28:06.142 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:28:06.257 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:28:06.302 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:28:06.347 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:28:06.388 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:28:06.451 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:28:06.684 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:28:06.738 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:28:06.811 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:28:06.922 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:28:06.962 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:28:07.001 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:28:07.068 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:28:07.118 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:28:07.179 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:28:07.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:28:07.436 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:28:07.492 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:28:07.529 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:28:07.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:28:07.712 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:28:07.762 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:28:07.807 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:28:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:28:07.876 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:28:08.060 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:28:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:28:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:28:08.322 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:28:08.368 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:28:08.427 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:28:08.487 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:28:08.528 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:28:08.573 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:28:08.654 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:28:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:28:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:28:08.925 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:28:08.987 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:28:09.030 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:28:09.100 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:28:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:28:09.309 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:28:10.256 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:28:10.309 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:28:10.342 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:28:10.464 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:28:10.570 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:28:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:28:10.727 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:28:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:28:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:28:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:28:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:28:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:28:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:28:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:28:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:28:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:28:14.545 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:28:14.585 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:28:14.739 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:28:14.775 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:28:14.833 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:28:15.366 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:28:15.396 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:28:15.521 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:28:15.548 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:28:18.337 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:28:18.387 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:28:19.036 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:28:19.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:28:19.278 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:28:19.278 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:28:19.279 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:28:19.279 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:19.316 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-04-24T22:28:32.088 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-04-24T22:28:32.089 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-04-24T22:28:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:32.305 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:28:32.354 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:28:32.354 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:32.354 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:28:32.354 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:34.016 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:28:34.095 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:28:34.096 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:34.096 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:28:34.096 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:28:34.096 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:28:34.096 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:34.358 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:28:34.427 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:34.908 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:28:34.999 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:28:35.000 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:35.000 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:28:35.000 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:28:35.000 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:28:35.000 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:35.126 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:28:35.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:28:35.261 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:28:35.492 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:28:35.546 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:28:35.631 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:28:35.748 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:28:35.913 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:28:35.975 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:28:36.021 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:28:36.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:28:36.667 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:28:36.706 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:28:36.758 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:28:36.861 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:28:37.159 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:28:38.191 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:28:38.243 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:28:39.338 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:28:39.368 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:28:39.513 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:28:39.700 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:28:39.728 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:28:39.823 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:39.879 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:28:39.916 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:28:40.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:28:40.145 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:28:41.207 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:28:41.240 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:28:41.327 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:41.383 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:28:41.475 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:28:41.475 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:41.476 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:28:41.476 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:41.542 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:28:41.617 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:28:41.617 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:41.617 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:28:41.617 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:28:41.618 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:28:41.618 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:42.009 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:28:42.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:47.039 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:28:47.083 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:28:47.154 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:28:47.199 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:28:47.248 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:28:47.593 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:28:47.644 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:28:47.672 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:28:48.552 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:28:48.603 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:28:48.659 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:28:49.960 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:28:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:28:50.176 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:28:50.356 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:28:50.400 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:28:50.400 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:28:50.526 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:28:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:28:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:28:50.688 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:28:50.830 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:28:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:28:51.120 INFO:teuthology.orchestra.run.smithi111.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T22:28:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:57.105 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:28:57.105 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:28:57.105 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:28:57.106 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:28:57.107 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:28:57.108 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:28:57.108 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:28:57.108 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:28:57.120 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:28:57.121 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:28:57.122 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:28:57.123 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:28:57.124 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:28:57.125 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:28:57.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:28:57.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:28:57.128 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:28:57.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:28:59.599 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.600 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.601 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:28:59.602 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:28:59.603 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:28:59.604 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:28:59.605 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:28:59.606 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:28:59.607 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:28:59.608 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:28:59.609 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:28:59.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:28:59.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:28:59.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:28:59.613 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:28:59.614 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-24T22:29:00.203 DEBUG:teuthology.parallel:result is None 2024-04-24T22:29:00.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:00.429 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:29:00.485 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.487 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.487 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:29:00.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:00.709 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:29:00.735 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.735 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.735 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:29:00.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:00.877 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:29:00.903 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.903 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:29:00.903 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:29:00.905 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T22:29:00.905 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:29:00.905 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:29:00.995 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:29:00.995 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:29:01.026 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:29:01.027 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:29:01.057 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T22:29:01.058 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:29:01.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:29:01.122 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:29:01.229 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:29:01.229 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:29:01.258 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:29:01.326 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:29:01.326 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:29:01.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:29:01.417 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T22:29:01.418 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:29:01.418 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:29:01.489 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:29:01.595 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:29:01.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:29:01.621 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:29:01.685 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:29:01.685 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:29:01.712 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:29:01.776 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T22:29:01.776 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:29:01.776 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:29:01.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:29:01.951 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:29:01.951 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:29:01.976 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:29:02.039 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:29:02.040 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:29:02.063 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:29:02.127 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T22:29:02.252 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T22:29:02.252 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:02.253 INFO:tasks.cephadm:Cluster fsid is 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:29:02.253 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T22:29:02.253 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.111', 'mon.b': '172.21.15.171', 'mon.c': '172.21.15.186'} 2024-04-24T22:29:02.254 INFO:tasks.cephadm:First mon is mon.a on smithi111 2024-04-24T22:29:02.254 INFO:tasks.cephadm:First mgr is x 2024-04-24T22:29:02.254 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T22:29:02.254 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-04-24T22:29:02.320 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-04-24T22:29:02.344 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-04-24T22:29:02.368 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T22:29:02.369 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:02.513 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T22:29:02.655 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:29:02.657 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T22:29:02.657 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:03.485 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 781263 Apr 24 22:29 /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:03.485 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:04.750 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 781263 Apr 24 22:29 /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:04.751 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:06.033 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 781263 Apr 24 22:29 /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:06.033 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:06.080 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:06.100 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T22:29:06.135 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T22:29:06.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T22:29:06.142 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T22:29:06.145 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T22:29:07.570 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T22:29:07.591 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T22:29:08.996 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T22:29:35.357 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-24T22:29:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T22:29:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T22:29:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-04-24T22:29:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T22:29:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-04-24T22:29:35.358 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-04-24T22:29:54.921 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:30:06.787 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:30:06.850 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-04-24T22:30:06.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-04-24T22:30:06.970 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-04-24T22:30:07.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-04-24T22:30:07.074 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-04-24T22:30:07.103 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-04-24T22:30:07.132 INFO:tasks.cephadm:Writing seed config... 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T22:30:07.134 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T22:30:07.135 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-24T22:30:07.136 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T22:30:07.137 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-24T22:30:07.138 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:30:07.138 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T22:30:07.179 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0d3ba9b8-028a-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T22:30:07.180 DEBUG:teuthology.orchestra.run.smithi111:mon.a> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T22:30:07.226 DEBUG:teuthology.orchestra.run.smithi111:mgr.x> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T22:30:07.272 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T22:30:07.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.111 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:30:07.537 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:30:07.537 INFO:teuthology.orchestra.run.smithi111.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '0d3ba9b8-028a-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.111', '--skip-admin-label'] 2024-04-24T22:30:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T22:30:07.538 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying podman|docker is present... 2024-04-24T22:30:07.607 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T22:30:07.608 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying lvm2 is present... 2024-04-24T22:30:07.608 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying time synchronization is in place... 2024-04-24T22:30:07.623 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T22:30:07.623 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T22:30:07.638 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T22:30:07.638 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:30:07.659 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout enabled 2024-04-24T22:30:07.680 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout active 2024-04-24T22:30:07.680 INFO:teuthology.orchestra.run.smithi111.stdout:Unit chronyd.service is enabled and running 2024-04-24T22:30:07.680 INFO:teuthology.orchestra.run.smithi111.stdout:Repeating the final host check... 2024-04-24T22:30:07.738 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T22:30:07.739 INFO:teuthology.orchestra.run.smithi111.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T22:30:07.739 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl is present 2024-04-24T22:30:07.739 INFO:teuthology.orchestra.run.smithi111.stdout:lvcreate is present 2024-04-24T22:30:07.758 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T22:30:07.758 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T22:30:07.784 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T22:30:07.785 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:30:07.809 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout enabled 2024-04-24T22:30:07.831 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout active 2024-04-24T22:30:07.832 INFO:teuthology.orchestra.run.smithi111.stdout:Unit chronyd.service is enabled and running 2024-04-24T22:30:07.832 INFO:teuthology.orchestra.run.smithi111.stdout:Host looks OK 2024-04-24T22:30:07.832 INFO:teuthology.orchestra.run.smithi111.stdout:Cluster fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:07.832 INFO:teuthology.orchestra.run.smithi111.stdout:Acquiring lock 139703473021856 on /run/cephadm/0d3ba9b8-028a-11ef-bc93-c7b262605968.lock 2024-04-24T22:30:07.833 INFO:teuthology.orchestra.run.smithi111.stdout:Lock 139703473021856 acquired on /run/cephadm/0d3ba9b8-028a-11ef-bc93-c7b262605968.lock 2024-04-24T22:30:07.834 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying IP 172.21.15.111 port 3300 ... 2024-04-24T22:30:07.835 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying IP 172.21.15.111 port 6789 ... 2024-04-24T22:30:07.836 INFO:teuthology.orchestra.run.smithi111.stdout:Base mon IP(s) is [172.21.15.111:3300, 172.21.15.111:6789], mon addrv is [v2:172.21.15.111:3300,v1:172.21.15.111:6789] 2024-04-24T22:30:07.852 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.111 metric 100 2024-04-24T22:30:07.852 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.111 metric 100 2024-04-24T22:30:07.865 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T22:30:07.865 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-24T22:30:07.866 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1753sec hoplimit 64 pref medium 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-04-24T22:30:07.878 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T22:30:07.880 INFO:teuthology.orchestra.run.smithi111.stdout:Mon IP `172.21.15.111` is in CIDR network `172.21.0.0/20` 2024-04-24T22:30:07.880 INFO:teuthology.orchestra.run.smithi111.stdout:Mon IP `172.21.15.111` is in CIDR network `172.21.0.0/20` 2024-04-24T22:30:07.881 INFO:teuthology.orchestra.run.smithi111.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T22:30:07.883 INFO:teuthology.orchestra.run.smithi111.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T22:30:07.886 INFO:teuthology.orchestra.run.smithi111.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T22:30:08.544 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T22:30:08.544 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T22:30:08.544 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T22:30:08.545 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T22:30:08.545 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T22:30:08.545 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T22:30:08.545 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T22:30:09.753 INFO:teuthology.orchestra.run.smithi111.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T22:30:09.754 INFO:teuthology.orchestra.run.smithi111.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T22:30:09.754 INFO:teuthology.orchestra.run.smithi111.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T22:30:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:stat: stdout 167 167 2024-04-24T22:30:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial keys... 2024-04-24T22:30:13.148 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQDzhylmp57hFhAAybTg4Mr35j4PeZigMHClvg== 2024-04-24T22:30:14.328 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQD1hylmNkMoJRAAsHnx1THxyYLWwB9D++2z7w== 2024-04-24T22:30:15.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQD2hylmVOzgMBAAGsxdTGnAmfLXk3VImp/QHg== 2024-04-24T22:30:15.422 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial monmap... 2024-04-24T22:30:16.575 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool for a [v2:172.21.15.111:3300,v1:172.21.15.111:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = quincy 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: set fsid to 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:30:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:Creating mon... 2024-04-24T22:30:17.666 INFO:teuthology.orchestra.run.smithi111.stdout:create mon.a on 2024-04-24T22:30:18.051 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T22:30:18.313 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T22:30:18.595 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target. 2024-04-24T22:30:18.595 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target. 2024-04-24T22:30:18.984 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a 2024-04-24T22:30:18.984 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T22:30:19.287 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target.wants/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@.service. 2024-04-24T22:30:19.969 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:19 smithi111 podman[27357]: 2024-04-24 22:30:19.594268792 +0000 UTC m=+0.036034068 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:30:19.970 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:19 smithi111 podman[27357]: 2024-04-24 22:30:19.965990694 +0000 UTC m=+0.407756045 container init a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T22:30:20.090 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:30:20.090 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-04-24T22:30:20.107 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:30:20.107 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:30:20.107 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-04-24T22:30:20.108 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T22:30:20.108 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mon to start... 2024-04-24T22:30:20.108 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mon... 2024-04-24T22:30:20.257 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:19 smithi111 podman[27357]: 2024-04-24 22:30:19.976681089 +0000 UTC m=+0.418446363 container start a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T22:30:20.258 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:20 smithi111 ceph-mon[27376]: mkfs 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:20.258 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:20 smithi111 bash[27357]: a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 2024-04-24T22:30:20.258 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:20 smithi111 ceph-mon[27376]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T22:30:20.258 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:20 smithi111 systemd[1]: Started Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: monmap epoch 1 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: last_changed 2024-04-24T22:30:15.899552+0000 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: min_mon_release 19 (squid) 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: election_strategy: 1 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: fsmap 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: mgrmap e1: no daemons active 2024-04-24T22:30:21.469 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:21 smithi111 ceph-mon[27376]: from='client.? 172.21.15.111:0/2912838046' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T22:30:21.476 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout id: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout services: 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.757361s) 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:21.483 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout data: 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.716+0000 7f7cf1400700 1 Processor -- start 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.717+0000 7f7cf1400700 1 -- start start 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.717+0000 7f7cf1400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.717+0000 7f7cf1400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec1115d0 con 0x7f7cec10eca0 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.718+0000 7f7cea800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.718+0000 7f7cea800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56508/0 (socket says 172.21.15.111:56508) 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.718+0000 7f7cea800700 1 -- 172.21.15.111:0/2014386154 learned_addr learned my addr 172.21.15.111:0/2014386154 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.718+0000 7f7cea800700 1 -- 172.21.15.111:0/2014386154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec111750 con 0x7f7cec10eca0 2024-04-24T22:30:21.484 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.719+0000 7f7cea800700 1 --2- 172.21.15.111:0/2014386154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098a0 tx=0x7f7cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e37ccf9ca08a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.719+0000 7f7ce9400700 1 -- 172.21.15.111:0/2014386154 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc004160 con 0x7f7cec10eca0 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.719+0000 7f7ce9400700 1 -- 172.21.15.111:0/2014386154 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc004300 con 0x7f7cec10eca0 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.719+0000 7f7ce9400700 1 -- 172.21.15.111:0/2014386154 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc0036a0 con 0x7f7cec10eca0 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 -- 172.21.15.111:0/2014386154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 msgr2=0x7f7cec111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 --2- 172.21.15.111:0/2014386154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098a0 tx=0x7f7cdc009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 -- 172.21.15.111:0/2014386154 shutdown_connections 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 --2- 172.21.15.111:0/2014386154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec111090 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 -- 172.21.15.111:0/2014386154 >> 172.21.15.111:0/2014386154 conn(0x7f7cec07d3a0 msgr2=0x7f7cec07f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 -- 172.21.15.111:0/2014386154 shutdown_connections 2024-04-24T22:30:21.485 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.720+0000 7f7cf1400700 1 -- 172.21.15.111:0/2014386154 wait complete. 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.721+0000 7f7cf1400700 1 Processor -- start 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cf1400700 1 -- start start 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cf1400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cf1400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec111c20 con 0x7f7cec10eca0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cea800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cea800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56516/0 (socket says 172.21.15.111:56516) 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.722+0000 7f7cea800700 1 -- 172.21.15.111:0/2912838046 learned_addr learned my addr 172.21.15.111:0/2912838046 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7cea800700 1 -- 172.21.15.111:0/2912838046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1a2080 con 0x7f7cec10eca0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7cea800700 1 --2- 172.21.15.111:0/2912838046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc003db0 con 0x7f7cec10eca0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc011630 con 0x7f7cec10eca0 2024-04-24T22:30:21.486 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc011950 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.723+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1a2310 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.724+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a5000 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.724+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cdc01a460 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.724+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7cdc01b780 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.725+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec063890 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.728+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7cdc01b980 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.807+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7cec02de30 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.808+0000 7f7ce3400700 1 -- 172.21.15.111:0/2912838046 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7cdc01bb60 con 0x7f7cec10eca0 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.810+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 msgr2=0x7f7cec1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.810+0000 7f7cf1400700 1 --2- 172.21.15.111:0/2912838046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).stop 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.810+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 shutdown_connections 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.810+0000 7f7cf1400700 1 --2- 172.21.15.111:0/2912838046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec10eca0 0x7f7cec1a1b40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.810+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 >> 172.21.15.111:0/2912838046 conn(0x7f7cec07d3a0 msgr2=0x7f7cec07f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.811+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 shutdown_connections 2024-04-24T22:30:21.487 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:20.811+0000 7f7cf1400700 1 -- 172.21.15.111:0/2912838046 wait complete. 2024-04-24T22:30:21.488 INFO:teuthology.orchestra.run.smithi111.stdout:mon is available 2024-04-24T22:30:21.488 INFO:teuthology.orchestra.run.smithi111.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T22:30:22.527 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:22 smithi111 ceph-mon[27376]: from='client.? 172.21.15.111:0/2973829817' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T22:30:22.527 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:22 smithi111 ceph-mon[27376]: from='client.? 172.21.15.111:0/2973829817' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T22:30:22.843 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout fsid = 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.111:3300,v1:172.21.15.111:6789] 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:22.844 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.050+0000 7f9301600700 1 Processor -- start 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.050+0000 7f9301600700 1 -- start start 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.050+0000 7f9301600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.050+0000 7f9301600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc07be80 con 0x7f92fc108880 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.051+0000 7f92faa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.051+0000 7f92faa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56528/0 (socket says 172.21.15.111:56528) 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.051+0000 7f92faa00700 1 -- 172.21.15.111:0/3161413355 learned_addr learned my addr 172.21.15.111:0/3161413355 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.051+0000 7f92faa00700 1 -- 172.21.15.111:0/3161413355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc10b1b0 con 0x7f92fc108880 2024-04-24T22:30:22.845 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.052+0000 7f92faa00700 1 --2- 172.21.15.111:0/3161413355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098a0 tx=0x7f92ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972674155e86f54d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.052+0000 7f92f9600700 1 -- 172.21.15.111:0/3161413355 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec004160 con 0x7f92fc108880 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.052+0000 7f92f9600700 1 -- 172.21.15.111:0/3161413355 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec004300 con 0x7f92fc108880 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.052+0000 7f92f9600700 1 -- 172.21.15.111:0/3161413355 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec0036a0 con 0x7f92fc108880 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 -- 172.21.15.111:0/3161413355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 msgr2=0x7f92fc10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 --2- 172.21.15.111:0/3161413355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098a0 tx=0x7f92ec009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 -- 172.21.15.111:0/3161413355 shutdown_connections 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 --2- 172.21.15.111:0/3161413355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc10ac70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 -- 172.21.15.111:0/3161413355 >> 172.21.15.111:0/3161413355 conn(0x7f92fc1022d0 msgr2=0x7f92fc1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 -- 172.21.15.111:0/3161413355 shutdown_connections 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.053+0000 7f9301600700 1 -- 172.21.15.111:0/3161413355 wait complete. 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.054+0000 7f9301600700 1 Processor -- start 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.054+0000 7f9301600700 1 -- start start 2024-04-24T22:30:22.846 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f9301600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f9301600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10b680 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f92faa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f92faa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56530/0 (socket says 172.21.15.111:56530) 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f92faa00700 1 -- 172.21.15.111:0/2973829817 learned_addr learned my addr 172.21.15.111:0/2973829817 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.055+0000 7f92faa00700 1 -- 172.21.15.111:0/2973829817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19dc70 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92faa00700 1 --2- 172.21.15.111:0/2973829817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec003db0 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec011630 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec011950 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19df00 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a0bf0 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f92ec011af0 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.056+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f92ec01c450 con 0x7f92fc108880 2024-04-24T22:30:22.847 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.057+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc063890 con 0x7f92fc108880 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.060+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f92ec032020 con 0x7f92fc108880 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.136+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f92fc02dcb0 con 0x7f92fc108880 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.143+0000 7f92f3400700 1 -- 172.21.15.111:0/2973829817 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f92ec004260 con 0x7f92fc108880 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.144+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 msgr2=0x7f92fc19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.144+0000 7f9301600700 1 --2- 172.21.15.111:0/2973829817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec004750 comp rx=0 tx=0).stop 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.144+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 shutdown_connections 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.144+0000 7f9301600700 1 --2- 172.21.15.111:0/2973829817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92fc108880 0x7f92fc19d730 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.144+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 >> 172.21.15.111:0/2973829817 conn(0x7f92fc1022d0 msgr2=0x7f92fc1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.145+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 shutdown_connections 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:22.145+0000 7f9301600700 1 -- 172.21.15.111:0/2973829817 wait complete. 2024-04-24T22:30:22.848 INFO:teuthology.orchestra.run.smithi111.stdout:Generating new minimal ceph.conf... 2024-04-24T22:30:23.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:23 smithi111 ceph-mon[27376]: from='client.? 172.21.15.111:0/1231039099' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:30:24.135 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.381+0000 7f3bfa600700 1 Processor -- start 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.381+0000 7f3bfa600700 1 -- start start 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.382+0000 7f3bfa600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.382+0000 7f3bfa600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf407c860 con 0x7f3bf407de30 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.382+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.382+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56540/0 (socket says 172.21.15.111:56540) 2024-04-24T22:30:24.136 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.382+0000 7f3bf3400700 1 -- 172.21.15.111:0/2801546942 learned_addr learned my addr 172.21.15.111:0/2801546942 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.383+0000 7f3bf3400700 1 -- 172.21.15.111:0/2801546942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf407c9e0 con 0x7f3bf407de30 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.383+0000 7f3bf3400700 1 --2- 172.21.15.111:0/2801546942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098a0 tx=0x7f3be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36df82153c2d564d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.383+0000 7f3bf2000700 1 -- 172.21.15.111:0/2801546942 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4004160 con 0x7f3bf407de30 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.383+0000 7f3bf2000700 1 -- 172.21.15.111:0/2801546942 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3be4004300 con 0x7f3bf407de30 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.383+0000 7f3bf2000700 1 -- 172.21.15.111:0/2801546942 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4011b50 con 0x7f3bf407de30 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 -- 172.21.15.111:0/2801546942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 msgr2=0x7f3bf407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 --2- 172.21.15.111:0/2801546942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098a0 tx=0x7f3be4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 -- 172.21.15.111:0/2801546942 shutdown_connections 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 --2- 172.21.15.111:0/2801546942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf407c290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 -- 172.21.15.111:0/2801546942 >> 172.21.15.111:0/2801546942 conn(0x7f3bf4103410 msgr2=0x7f3bf4105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 -- 172.21.15.111:0/2801546942 shutdown_connections 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.384+0000 7f3bfa600700 1 -- 172.21.15.111:0/2801546942 wait complete. 2024-04-24T22:30:24.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.385+0000 7f3bfa600700 1 Processor -- start 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bfa600700 1 -- start start 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bfa600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bfa600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410e400 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56552/0 (socket says 172.21.15.111:56552) 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bf3400700 1 -- 172.21.15.111:0/1231039099 learned_addr learned my addr 172.21.15.111:0/1231039099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.386+0000 7f3bf3400700 1 -- 172.21.15.111:0/1231039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419e540 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bf3400700 1 --2- 172.21.15.111:0/1231039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3be4011b60 tx=0x7f3be4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4003970 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3be4003b10 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419e7d0 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4024610 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a14c0 con 0x7f3bf407de30 2024-04-24T22:30:24.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.387+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3be4024c50 con 0x7f3bf407de30 2024-04-24T22:30:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.388+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3be4011e00 con 0x7f3bf407de30 2024-04-24T22:30:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.388+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf4063890 con 0x7f3bf407de30 2024-04-24T22:30:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.391+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3be4043dd0 con 0x7f3bf407de30 2024-04-24T22:30:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.468+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3bf41a1690 con 0x7f3bf407de30 2024-04-24T22:30:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.469+0000 7f3bebe00700 1 -- 172.21.15.111:0/1231039099 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f3be4032030 con 0x7f3bf407de30 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 msgr2=0x7f3bf419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 --2- 172.21.15.111:0/1231039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3be4011b60 tx=0x7f3be4011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 shutdown_connections 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 --2- 172.21.15.111:0/1231039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf407de30 0x7f3bf419e000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 >> 172.21.15.111:0/1231039099 conn(0x7f3bf4103410 msgr2=0x7f3bf4103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 shutdown_connections 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:23.470+0000 7f3bfa600700 1 -- 172.21.15.111:0/1231039099 wait complete. 2024-04-24T22:30:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:Restarting the monitor... 2024-04-24T22:30:24.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:24 smithi111 systemd[1]: Stopping Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:30:24.953 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a[27372]: 2024-04-24T22:30:24.682+0000 7f19e2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:30:24.954 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a[27372]: 2024-04-24T22:30:24.682+0000 7f19e2400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T22:30:25.266 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:24 smithi111 podman[27622]: 2024-04-24 22:30:24.951070924 +0000 UTC m=+0.382871238 container died a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:30:25.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:25 smithi111 podman[27622]: 2024-04-24 22:30:25.417881365 +0000 UTC m=+0.849681705 container cleanup a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T22:30:25.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:25 smithi111 bash[27622]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a 2024-04-24T22:30:26.020 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:25 smithi111 podman[27634]: 2024-04-24 22:30:25.635361276 +0000 UTC m=+0.669686643 container remove a57ce75fbdbbd5168d0aa03b1883ac45ec5156ed983f65a2814d150dd789c086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T22:30:26.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T22:30:26.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 systemd[1]: Stopped Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:30:26.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.907s CPU time. 2024-04-24T22:30:26.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 systemd[1]: Starting Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:30:26.732 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 podman[27727]: 2024-04-24 22:30:26.344039945 +0000 UTC m=+0.037001310 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:30:26.732 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 podman[27727]: 2024-04-24 22:30:26.444885178 +0000 UTC m=+0.137846576 container create e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:30:26.732 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 podman[27727]: 2024-04-24 22:30:26.673344058 +0000 UTC m=+0.366305480 container init e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T22:30:26.732 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 podman[27727]: 2024-04-24 22:30:26.686294043 +0000 UTC m=+0.379255451 container start e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T22:30:26.786 INFO:teuthology.orchestra.run.smithi111.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T22:30:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T22:30:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: pidfile_write: ignore empty --pid-file 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: load: jerasure load: lrc 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: RocksDB version: 7.9.2 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Git sha 0 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: DB SUMMARY 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: DB Session ID: G7DXBW1SS1VSACEVA7GL 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: CURRENT file: CURRENT 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106178 ; 2024-04-24T22:30:27.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.error_if_exists: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.create_if_missing: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.paranoid_checks: 1 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.env: 0x5633ce2e7c60 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.info_log: 0x5633e1589ae0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.statistics: (nil) 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.use_fsync: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_log_file_size: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T22:30:27.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_fallocate: 1 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.db_log_dir: 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.wal_dir: 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:30:27.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.write_buffer_manager: 0x5633e155c1e0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:30:27.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.unordered_write: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.row_cache: None 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.wal_filter: None 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.wal_compression: 0 2024-04-24T22:30:27.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:30:27.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:30:27.042 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_open_files: -1 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Compression algorithms supported: 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kZSTD supported: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:30:27.043 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.merge_operator: 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_filter: None 2024-04-24T22:30:27.044 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5633e1589be0) 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_type: 0 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: data_block_index_type: 0 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_shortening: 1 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: checksum: 4 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: no_block_cache: 0 2024-04-24T22:30:27.045 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache: 0x5633e158d090 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_options: 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: capacity : 536870912 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: num_shard_bits : 4 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: strict_capacity_limit : 0 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_compressed: (nil) 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: persistent_cache: (nil) 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_size: 4096 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_size_deviation: 10 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_restart_interval: 16 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_block_restart_interval: 1 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: metadata_block_size: 4096 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: partition_filters: 0 2024-04-24T22:30:27.046 INFO:journalctl@ceph.mon.a.smithi111.stdout: use_delta_encoding: 1 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: filter_policy: bloomfilter 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: whole_key_filtering: 1 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: verify_compression: 0 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: format_version: 5 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: enable_index_compression: 1 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_align: 0 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: max_auto_readahead_size: 262144 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: prepopulate_block_cache: 0 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: initial_auto_readahead_size: 8192 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression: NoCompression 2024-04-24T22:30:27.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.num_levels: 7 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:30:27.048 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:30:27.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:30:27.050 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:30:27.051 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:30:27.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:30:27.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.ttl: 2592000 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:30:27.054 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 722cd5bf-99df-4f82-8a88-e8068a4d6dbd 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997826743096, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997826745865, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99027, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14524, "raw_average_key_size": 49, "raw_value_size": 91189, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713997826, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "722cd5bf-99df-4f82-8a88-e8068a4d6dbd", "db_session_id": "G7DXBW1SS1VSACEVA7GL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997826746156, "job": 1, "event": "recovery_finished"} 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5633e167e000 2024-04-24T22:30:27.055 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: DB pointer 0x5633e1668000 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** DB Stats ** 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:30:27.056 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** Compaction Stats [default] ** 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** Compaction Stats [default] ** 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:30:27.057 INFO:journalctl@ceph.mon.a.smithi111.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 58.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative compaction: 0.00 GB write, 10.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval compaction: 0.00 GB write, 10.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:30:27.058 INFO:journalctl@ceph.mon.a.smithi111.stdout: Block cache BinnedLRUCache@0x5633e158d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: starting mon.a rank 0 at public addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] at bind addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???) e1 preinit fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).mds e1 new map 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).mds e1 print_map 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: e1 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:30:27.059 INFO:journalctl@ceph.mon.a.smithi111.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout: legacy client fscid: -1 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout: No filesystems configured 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 bash[27727]: e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 systemd[1]: Started Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T22:30:27.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: monmap epoch 1 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: last_changed 2024-04-24T22:30:15.899552+0000 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: min_mon_release 19 (squid) 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: election_strategy: 1 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: fsmap 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T22:30:27.061 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:26 smithi111 ceph-mon[27746]: mgrmap e1: no daemons active 2024-04-24T22:30:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.386+0000 7faac4c00700 1 Processor -- start 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.386+0000 7faac4c00700 1 -- start start 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.387+0000 7faac4c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.387+0000 7faac4c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faac007c8c0 con 0x7faac007de90 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.388+0000 7faab5e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.388+0000 7faab5e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33022/0 (socket says 172.21.15.111:33022) 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.388+0000 7faab5e00700 1 -- 172.21.15.111:0/778980749 learned_addr learned my addr 172.21.15.111:0/778980749 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.389+0000 7faab5e00700 1 -- 172.21.15.111:0/778980749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac007ca40 con 0x7faac007de90 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.389+0000 7faab5e00700 1 --2- 172.21.15.111:0/778980749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faab00098a0 tx=0x7faab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36ce0c3f4810013 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.389+0000 7faabd400700 1 -- 172.21.15.111:0/778980749 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab0004160 con 0x7faac007de90 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.389+0000 7faabd400700 1 -- 172.21.15.111:0/778980749 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faab0004300 con 0x7faac007de90 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.389+0000 7faabd400700 1 -- 172.21.15.111:0/778980749 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab0011b50 con 0x7faac007de90 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.390+0000 7faac4c00700 1 -- 172.21.15.111:0/778980749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 msgr2=0x7faac007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.390+0000 7faac4c00700 1 --2- 172.21.15.111:0/778980749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faab00098a0 tx=0x7faab0009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.390+0000 7faac4c00700 1 -- 172.21.15.111:0/778980749 shutdown_connections 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.390+0000 7faac4c00700 1 --2- 172.21.15.111:0/778980749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac007c2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.390+0000 7faac4c00700 1 -- 172.21.15.111:0/778980749 >> 172.21.15.111:0/778980749 conn(0x7faac0103410 msgr2=0x7faac0105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.391+0000 7faac4c00700 1 -- 172.21.15.111:0/778980749 shutdown_connections 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.391+0000 7faac4c00700 1 -- 172.21.15.111:0/778980749 wait complete. 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.392+0000 7faac4c00700 1 Processor -- start 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.392+0000 7faac4c00700 1 -- start start 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faac4c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faac4c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faac010e400 con 0x7faac007de90 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faab5e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faab5e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33024/0 (socket says 172.21.15.111:33024) 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faab5e00700 1 -- 172.21.15.111:0/2739802785 learned_addr learned my addr 172.21.15.111:0/2739802785 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.393+0000 7faab5e00700 1 -- 172.21.15.111:0/2739802785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac019e570 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faab5e00700 1 --2- 172.21.15.111:0/2739802785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fb0 tx=0x7faab0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab00038b0 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faab0011e50 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac019e800 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a14f0 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.394+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab0019550 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.395+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faab0019be0 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.395+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faab0024730 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.396+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac0063890 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.399+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7faab0003a50 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.477+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faac01a1760 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.480+0000 7faab7400700 1 -- 172.21.15.111:0/2739802785 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faab0024930 con 0x7faac007de90 2024-04-24T22:30:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 msgr2=0x7faac019e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 --2- 172.21.15.111:0/2739802785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fb0 tx=0x7faab0011bf0 comp rx=0 tx=0).stop 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 shutdown_connections 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 --2- 172.21.15.111:0/2739802785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac007de90 0x7faac019e030 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 >> 172.21.15.111:0/2739802785 conn(0x7faac0103410 msgr2=0x7faac0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 shutdown_connections 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:27.481+0000 7faac4c00700 1 -- 172.21.15.111:0/2739802785 wait complete. 2024-04-24T22:30:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T22:30:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:30:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:Creating mgr... 2024-04-24T22:30:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T22:30:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T22:30:28.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:28 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2739802785' entity='client.admin' 2024-04-24T22:30:28.716 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x 2024-04-24T22:30:28.716 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T22:30:28.973 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968.target.wants/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@.service. 2024-04-24T22:30:29.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 systemd[1]: Starting Ceph mgr.x for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:30:29.628 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 podman[27974]: 2024-04-24 22:30:29.256929163 +0000 UTC m=+0.032579329 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:30:29.628 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 podman[27974]: 2024-04-24 22:30:29.380965224 +0000 UTC m=+0.156615437 container create a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T22:30:29.628 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 podman[27974]: 2024-04-24 22:30:29.626379232 +0000 UTC m=+0.402029458 container init a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:30:29.749 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:30:29.749 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-04-24T22:30:29.765 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:30:29.765 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:30:29.765 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-04-24T22:30:29.765 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T22:30:29.785 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:30:29.785 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr to start... 2024-04-24T22:30:29.806 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr... 2024-04-24T22:30:29.896 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 podman[27974]: 2024-04-24 22:30:29.636498928 +0000 UTC m=+0.412149101 container start a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:30:29.896 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 bash[27974]: a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e 2024-04-24T22:30:29.896 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 systemd[1]: Started Ceph mgr.x for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:30:30.151 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:29.893+0000 7fe13c320200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:30:30.425 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:30.153+0000 7fe13c320200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:30:30.426 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:30.286+0000 7fe13c320200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:30:30.708 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:30 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/637934932' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:30:30.960 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:30.705+0000 7fe13c320200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:30:30.960 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:30.821+0000 7fe13c320200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:30:31.258 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:31.271 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:31.271 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:30:31.271 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:30:31.271 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T22:30:31.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:30:31.273 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:30:31.274 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.275 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:30:20.053855+0000", 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:31.276 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.423+0000 7f2658e00700 1 Processor -- start 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.423+0000 7f2658e00700 1 -- start start 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.424+0000 7f2658e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.424+0000 7f2658e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f265407c7f0 con 0x7f265407de00 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.424+0000 7f2653400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.424+0000 7f2653400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33054/0 (socket says 172.21.15.111:33054) 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.424+0000 7f2653400700 1 -- 172.21.15.111:0/734819947 learned_addr learned my addr 172.21.15.111:0/734819947 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:31.277 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.425+0000 7f2653400700 1 -- 172.21.15.111:0/734819947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265407c970 con 0x7f265407de00 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.425+0000 7f2653400700 1 --2- 172.21.15.111:0/734819947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26440098a0 tx=0x7f2644009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3675174628c2890a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.426+0000 7f2652000700 1 -- 172.21.15.111:0/734819947 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2644004160 con 0x7f265407de00 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.426+0000 7f2652000700 1 -- 172.21.15.111:0/734819947 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2644004300 con 0x7f265407de00 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.426+0000 7f2652000700 1 -- 172.21.15.111:0/734819947 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2644011b50 con 0x7f265407de00 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.426+0000 7f2658e00700 1 -- 172.21.15.111:0/734819947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 msgr2=0x7f265407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.426+0000 7f2658e00700 1 --2- 172.21.15.111:0/734819947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26440098a0 tx=0x7f2644009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.427+0000 7f2658e00700 1 -- 172.21.15.111:0/734819947 shutdown_connections 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.427+0000 7f2658e00700 1 --2- 172.21.15.111:0/734819947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265407c2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.427+0000 7f2658e00700 1 -- 172.21.15.111:0/734819947 >> 172.21.15.111:0/734819947 conn(0x7f2654103210 msgr2=0x7f2654105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:31.278 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.427+0000 7f2658e00700 1 -- 172.21.15.111:0/734819947 shutdown_connections 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.427+0000 7f2658e00700 1 -- 172.21.15.111:0/734819947 wait complete. 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.428+0000 7f2658e00700 1 Processor -- start 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.428+0000 7f2658e00700 1 -- start start 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.428+0000 7f2658e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.428+0000 7f2658e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f265410e220 con 0x7f265407de00 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.429+0000 7f2653400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.429+0000 7f2653400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33068/0 (socket says 172.21.15.111:33068) 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.429+0000 7f2653400700 1 -- 172.21.15.111:0/637934932 learned_addr learned my addr 172.21.15.111:0/637934932 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.429+0000 7f2653400700 1 -- 172.21.15.111:0/637934932 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265419e390 con 0x7f265407de00 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.429+0000 7f2653400700 1 --2- 172.21.15.111:0/637934932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2644011b60 tx=0x7f2644011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:31.279 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2644003970 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2644003b10 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265419e620 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2644024610 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541a1310 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2644024ca0 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.430+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2644011e00 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.431+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2654063890 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.434+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f264401a9f0 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.513+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f26541a14e0 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.514+0000 7f264be00700 1 -- 172.21.15.111:0/637934932 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f264401a430 con 0x7f265407de00 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 msgr2=0x7f265419de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 --2- 172.21.15.111:0/637934932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2644011b60 tx=0x7f2644011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 shutdown_connections 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 --2- 172.21.15.111:0/637934932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f265407de00 0x7f265419de50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 >> 172.21.15.111:0/637934932 conn(0x7f2654103210 msgr2=0x7f2654103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:31.280 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 shutdown_connections 2024-04-24T22:30:31.281 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:30.515+0000 7f2658e00700 1 -- 172.21.15.111:0/637934932 wait complete. 2024-04-24T22:30:31.281 INFO:teuthology.orchestra.run.smithi111.stdout:mgr not available, waiting (1/15)... 2024-04-24T22:30:32.783 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:32.383+0000 7fe13c320200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:30:32.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:32.607+0000 7fe13c320200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:30:33.197 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:32.820+0000 7fe13c320200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:30:33.474 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:33.195+0000 7fe13c320200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:30:33.475 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:33.320+0000 7fe13c320200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:30:33.756 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:33.586+0000 7fe13c320200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:30:34.015 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:33.844+0000 7fe13c320200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:30:34.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:34 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/368444058' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T22:30:34.691 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:30:34.691 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.691 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:30:34.692 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:30:34.700 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:34.701 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:30:20.053855+0000", 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.906+0000 7f9fc1200700 1 Processor -- start 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.906+0000 7f9fc1200700 1 -- start start 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.907+0000 7f9fc1200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.907+0000 7f9fc1200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc107700 con 0x7f9fbc106d10 2024-04-24T22:30:34.702 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.907+0000 7f9fb9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.907+0000 7f9fb9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33072/0 (socket says 172.21.15.111:33072) 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.907+0000 7f9fb9e00700 1 -- 172.21.15.111:0/1929449473 learned_addr learned my addr 172.21.15.111:0/1929449473 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.908+0000 7f9fb9e00700 1 -- 172.21.15.111:0/1929449473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc107f10 con 0x7f9fbc106d10 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.908+0000 7f9fb9e00700 1 --2- 172.21.15.111:0/1929449473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098a0 tx=0x7f9fac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d52ae33690c6dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.908+0000 7f9fb8a00700 1 -- 172.21.15.111:0/1929449473 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac004160 con 0x7f9fbc106d10 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fb8a00700 1 -- 172.21.15.111:0/1929449473 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fac004300 con 0x7f9fbc106d10 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fb8a00700 1 -- 172.21.15.111:0/1929449473 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac011b50 con 0x7f9fbc106d10 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fc1200700 1 -- 172.21.15.111:0/1929449473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 msgr2=0x7f9fbc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fc1200700 1 --2- 172.21.15.111:0/1929449473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098a0 tx=0x7f9fac009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fc1200700 1 -- 172.21.15.111:0/1929449473 shutdown_connections 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fc1200700 1 --2- 172.21.15.111:0/1929449473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc107130 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.909+0000 7f9fc1200700 1 -- 172.21.15.111:0/1929449473 >> 172.21.15.111:0/1929449473 conn(0x7f9fbc1022d0 msgr2=0x7f9fbc1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.910+0000 7f9fc1200700 1 -- 172.21.15.111:0/1929449473 shutdown_connections 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.910+0000 7f9fc1200700 1 -- 172.21.15.111:0/1929449473 wait complete. 2024-04-24T22:30:34.703 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fc1200700 1 Processor -- start 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fc1200700 1 -- start start 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fc1200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fc1200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc10b2c0 con 0x7f9fbc106d10 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fb9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.911+0000 7f9fb9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33086/0 (socket says 172.21.15.111:33086) 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb9e00700 1 -- 172.21.15.111:0/368444058 learned_addr learned my addr 172.21.15.111:0/368444058 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb9e00700 1 -- 172.21.15.111:0/368444058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc19c3d0 con 0x7f9fbc106d10 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb9e00700 1 --2- 172.21.15.111:0/368444058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009fb0 tx=0x7f9fac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac0038b0 con 0x7f9fbc106d10 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fac011e50 con 0x7f9fbc106d10 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac019550 con 0x7f9fbc106d10 2024-04-24T22:30:34.704 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.912+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc19c660 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.913+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc19f350 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.913+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9fac019be0 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.913+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fac024960 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.914+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc063890 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.916+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9fac003a50 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.994+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9fbc0008d0 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.995+0000 7f9fb2a00700 1 -- 172.21.15.111:0/368444058 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9fac032030 con 0x7f9fbc106d10 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.996+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 msgr2=0x7f9fbc19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.996+0000 7f9fc1200700 1 --2- 172.21.15.111:0/368444058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009fb0 tx=0x7f9fac011bf0 comp rx=0 tx=0).stop 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.997+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 shutdown_connections 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.997+0000 7f9fc1200700 1 --2- 172.21.15.111:0/368444058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fbc106d10 0x7f9fbc19be90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.997+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 >> 172.21.15.111:0/368444058 conn(0x7f9fbc1022d0 msgr2=0x7f9fbc1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.997+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 shutdown_connections 2024-04-24T22:30:34.705 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:33.997+0000 7f9fc1200700 1 -- 172.21.15.111:0/368444058 wait complete. 2024-04-24T22:30:34.706 INFO:teuthology.orchestra.run.smithi111.stdout:mgr not available, waiting (2/15)... 2024-04-24T22:30:35.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:34.642+0000 7fe13c320200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:30:35.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:35.186+0000 7fe13c320200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:30:35.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:35.311+0000 7fe13c320200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:30:36.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:35.572+0000 7fe13c320200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:30:36.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:36.237+0000 7fe13c320200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:30:36.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:36.356+0000 7fe13c320200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:30:36.974 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:36.666+0000 7fe13c320200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:30:36.974 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:36.845+0000 7fe13c320200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:30:36.974 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:36.972+0000 7fe13c320200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:30:37.241 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:37.127+0000 7fe13c320200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:30:37.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:37 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2412290815' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:30:37.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:37.251+0000 7fe13c320200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:30:38.140 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:30:38.141 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:30:38.142 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:30:38.151 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:30:38.151 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:30:38.152 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:30:38.153 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:30:20.053855+0000", 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.294+0000 7f3e9fa00700 1 Processor -- start 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.295+0000 7f3e9fa00700 1 -- start start 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.295+0000 7f3e9fa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.295+0000 7f3e9fa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e98108770 con 0x7f3e98107e10 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.295+0000 7f3e9d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.296+0000 7f3e9d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34440/0 (socket says 172.21.15.111:34440) 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.296+0000 7f3e9d000700 1 -- 172.21.15.111:0/1218470222 learned_addr learned my addr 172.21.15.111:0/1218470222 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.296+0000 7f3e9d000700 1 -- 172.21.15.111:0/1218470222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e98108f70 con 0x7f3e98107e10 2024-04-24T22:30:38.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.296+0000 7f3e9d000700 1 --2- 172.21.15.111:0/1218470222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0098a0 tx=0x7f3e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33f17d77972df6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.296+0000 7f3e97400700 1 -- 172.21.15.111:0/1218470222 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c004160 con 0x7f3e98107e10 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.297+0000 7f3e97400700 1 -- 172.21.15.111:0/1218470222 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e8c004300 con 0x7f3e98107e10 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.297+0000 7f3e97400700 1 -- 172.21.15.111:0/1218470222 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c011b50 con 0x7f3e98107e10 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.297+0000 7f3e9fa00700 1 -- 172.21.15.111:0/1218470222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 msgr2=0x7f3e98108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.297+0000 7f3e9fa00700 1 --2- 172.21.15.111:0/1218470222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0098a0 tx=0x7f3e8c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.298+0000 7f3e9fa00700 1 -- 172.21.15.111:0/1218470222 shutdown_connections 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.298+0000 7f3e9fa00700 1 --2- 172.21.15.111:0/1218470222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e98108230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.298+0000 7f3e9fa00700 1 -- 172.21.15.111:0/1218470222 >> 172.21.15.111:0/1218470222 conn(0x7f3e98103410 msgr2=0x7f3e98105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.298+0000 7f3e9fa00700 1 -- 172.21.15.111:0/1218470222 shutdown_connections 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.298+0000 7f3e9fa00700 1 -- 172.21.15.111:0/1218470222 wait complete. 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.299+0000 7f3e9fa00700 1 Processor -- start 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.299+0000 7f3e9fa00700 1 -- start start 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9fa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9fa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9810c3b0 con 0x7f3e98107e10 2024-04-24T22:30:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34446/0 (socket says 172.21.15.111:34446) 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9d000700 1 -- 172.21.15.111:0/2412290815 learned_addr learned my addr 172.21.15.111:0/2412290815 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.300+0000 7f3e9d000700 1 -- 172.21.15.111:0/2412290815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9819e560 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e9d000700 1 --2- 172.21.15.111:0/2412290815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c011b60 tx=0x7f3e8c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c003970 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e8c003b10 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c024610 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9819e7f0 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.301+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e981a14e0 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.302+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e8c024c50 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.302+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e8c011e00 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.303+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e98063890 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.305+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3e8c01a9f0 con 0x7f3e98107e10 2024-04-24T22:30:38.156 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.386+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3e981a16b0 con 0x7f3e98107e10 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.386+0000 7f3e95600700 1 -- 172.21.15.111:0/2412290815 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f3e8c01a430 con 0x7f3e98107e10 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.388+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 msgr2=0x7f3e9819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.388+0000 7f3e9fa00700 1 --2- 172.21.15.111:0/2412290815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c011b60 tx=0x7f3e8c011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.389+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 shutdown_connections 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.389+0000 7f3e9fa00700 1 --2- 172.21.15.111:0/2412290815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e98107e10 0x7f3e9819e020 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.389+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 >> 172.21.15.111:0/2412290815 conn(0x7f3e98103410 msgr2=0x7f3e98103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.389+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 shutdown_connections 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:37.389+0000 7f3e9fa00700 1 -- 172.21.15.111:0/2412290815 wait complete. 2024-04-24T22:30:38.157 INFO:teuthology.orchestra.run.smithi111.stdout:mgr not available, waiting (3/15)... 2024-04-24T22:30:38.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:38.267+0000 7fe13c320200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:30:39.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:38.546+0000 7fe13c320200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: Activating manager daemon x 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: mgrmap e2: x(active, starting, since 0.011122s) 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:30:39.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: Manager daemon x is now available 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' 2024-04-24T22:30:39.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:38 smithi111 ceph-mon[27746]: from='mgr.14100 172.21.15.111:0/862699951' entity='mgr.x' 2024-04-24T22:30:41.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:40 smithi111 ceph-mon[27746]: mgrmap e3: x(active, since 1.02172s) 2024-04-24T22:30:41.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:41 smithi111 ceph-mon[27746]: mgrmap e4: x(active, since 2s) 2024-04-24T22:30:41.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:41 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1293818430' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:30:41.825 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:41.826 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:30:41.827 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:30:41.828 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:30:20.053855+0000", 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.781+0000 7fdf78600700 1 Processor -- start 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.781+0000 7fdf78600700 1 -- start start 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.781+0000 7fdf78600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.781+0000 7fdf78600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7010aa00 con 0x7fdf7010a010 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.782+0000 7fdf75c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:41.829 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.782+0000 7fdf75c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34534/0 (socket says 172.21.15.111:34534) 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.782+0000 7fdf75c00700 1 -- 172.21.15.111:0/4222167580 learned_addr learned my addr 172.21.15.111:0/4222167580 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.782+0000 7fdf75c00700 1 -- 172.21.15.111:0/4222167580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7010b1c0 con 0x7fdf7010a010 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.783+0000 7fdf75c00700 1 --2- 172.21.15.111:0/4222167580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c149bad999f0a620 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.783+0000 7fdf6fe00700 1 -- 172.21.15.111:0/4222167580 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf64004160 con 0x7fdf7010a010 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.783+0000 7fdf6fe00700 1 -- 172.21.15.111:0/4222167580 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf64004300 con 0x7fdf7010a010 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.783+0000 7fdf6fe00700 1 -- 172.21.15.111:0/4222167580 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf64011b50 con 0x7fdf7010a010 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 -- 172.21.15.111:0/4222167580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 msgr2=0x7fdf7010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 --2- 172.21.15.111:0/4222167580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 -- 172.21.15.111:0/4222167580 shutdown_connections 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 --2- 172.21.15.111:0/4222167580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7010a430 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 -- 172.21.15.111:0/4222167580 >> 172.21.15.111:0/4222167580 conn(0x7fdf7007d3a0 msgr2=0x7fdf7007d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 -- 172.21.15.111:0/4222167580 shutdown_connections 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.784+0000 7fdf78600700 1 -- 172.21.15.111:0/4222167580 wait complete. 2024-04-24T22:30:41.830 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.785+0000 7fdf78600700 1 Processor -- start 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.785+0000 7fdf78600700 1 -- start start 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.785+0000 7fdf78600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.785+0000 7fdf78600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7010e570 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf75c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf75c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34536/0 (socket says 172.21.15.111:34536) 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf75c00700 1 -- 172.21.15.111:0/1293818430 learned_addr learned my addr 172.21.15.111:0/1293818430 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf75c00700 1 -- 172.21.15.111:0/1293818430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7019e560 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf75c00700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf64011850 tx=0x7fdf64011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf64003af0 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf64020070 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.786+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf640195a0 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.787+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7019e7f0 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.787+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf701a14e0 con 0x7fdf7010a010 2024-04-24T22:30:41.831 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.788+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fdf64019850 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.788+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf70063890 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.789+0000 7fdf6e000700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 0x7fdf5803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.789+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf64011e00 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.789+0000 7fdf75200700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 0x7fdf5803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.790+0000 7fdf75200700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 0x7fdf5803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006fd0 tx=0x7fdf5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:40.795+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf6400e260 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.105+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdf701a16b0 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.106+0000 7fdf6e000700 1 -- 172.21.15.111:0/1293818430 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdf701a16b0 con 0x7fdf7010a010 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 msgr2=0x7fdf5803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 0x7fdf5803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006fd0 tx=0x7fdf5c006e40 comp rx=0 tx=0).stop 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 msgr2=0x7fdf7019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf64011850 tx=0x7fdf64011c10 comp rx=0 tx=0).stop 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 shutdown_connections 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fdf58039050 0x7fdf5803b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 --2- 172.21.15.111:0/1293818430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf7010a010 0x7fdf7019e020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:41.833 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 >> 172.21.15.111:0/1293818430 conn(0x7fdf7007d3a0 msgr2=0x7fdf701075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:41.833 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 shutdown_connections 2024-04-24T22:30:41.833 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:41.109+0000 7fdf78600700 1 -- 172.21.15.111:0/1293818430 wait complete. 2024-04-24T22:30:41.833 INFO:teuthology.orchestra.run.smithi111.stdout:mgr is available 2024-04-24T22:30:43.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:42 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2388830147' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T22:30:43.374 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:43.380 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T22:30:43.380 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout fsid = 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:43.380 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T22:30:43.380 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T22:30:43.381 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.432+0000 7f13fb200700 1 Processor -- start 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.432+0000 7f13fb200700 1 -- start start 2024-04-24T22:30:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.433+0000 7f13fb200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:43.384 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.433+0000 7f13fb200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10a880 con 0x7f13fc109e90 2024-04-24T22:30:43.384 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.433+0000 7f13f9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:43.384 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.433+0000 7f13f9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34552/0 (socket says 172.21.15.111:34552) 2024-04-24T22:30:43.384 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.433+0000 7f13f9e00700 1 -- 172.21.15.111:0/2306103997 learned_addr learned my addr 172.21.15.111:0/2306103997 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:43.384 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.434+0000 7f13f9e00700 1 -- 172.21.15.111:0/2306103997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc10aff0 con 0x7f13fc109e90 2024-04-24T22:30:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.434+0000 7f13f9e00700 1 --2- 172.21.15.111:0/2306103997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0098a0 tx=0x7f13ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e08d33f52a74160 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.434+0000 7f13f8a00700 1 -- 172.21.15.111:0/2306103997 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec004160 con 0x7f13fc109e90 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13f8a00700 1 -- 172.21.15.111:0/2306103997 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13ec004300 con 0x7f13fc109e90 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13f8a00700 1 -- 172.21.15.111:0/2306103997 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec011b50 con 0x7f13fc109e90 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13fb200700 1 -- 172.21.15.111:0/2306103997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 msgr2=0x7f13fc10a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13fb200700 1 --2- 172.21.15.111:0/2306103997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0098a0 tx=0x7f13ec009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13fb200700 1 -- 172.21.15.111:0/2306103997 shutdown_connections 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13fb200700 1 --2- 172.21.15.111:0/2306103997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc10a2b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.435+0000 7f13fb200700 1 -- 172.21.15.111:0/2306103997 >> 172.21.15.111:0/2306103997 conn(0x7f13fc07d260 msgr2=0x7f13fc07d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.436+0000 7f13fb200700 1 -- 172.21.15.111:0/2306103997 shutdown_connections 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.436+0000 7f13fb200700 1 -- 172.21.15.111:0/2306103997 wait complete. 2024-04-24T22:30:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.437+0000 7f13fb200700 1 Processor -- start 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.437+0000 7f13fb200700 1 -- start start 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.437+0000 7f13fb200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.437+0000 7f13fb200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10e3a0 con 0x7f13fc109e90 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.438+0000 7f13f9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.438+0000 7f13f9e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34554/0 (socket says 172.21.15.111:34554) 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.438+0000 7f13f9e00700 1 -- 172.21.15.111:0/2388830147 learned_addr learned my addr 172.21.15.111:0/2388830147 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.438+0000 7f13f9e00700 1 -- 172.21.15.111:0/2388830147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc19e3b0 con 0x7f13fc109e90 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.438+0000 7f13f9e00700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13ec011850 tx=0x7f13ec011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.439+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec003af0 con 0x7f13fc109e90 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.439+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13ec020070 con 0x7f13fc109e90 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.439+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec0195a0 con 0x7f13fc109e90 2024-04-24T22:30:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.439+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc19e640 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.439+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1a1330 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.440+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f13ec019850 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.440+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc063890 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.441+0000 7f13f2a00700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 0x7f13e003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.441+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f13ec011e00 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.441+0000 7f13f9400700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 0x7f13e003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.442+0000 7f13f9400700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 0x7f13e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13e8006fd0 tx=0x7f13e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.446+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f13ec00e260 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.667+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f13fc1a1500 con 0x7f13fc109e90 2024-04-24T22:30:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.670+0000 7f13f2a00700 1 -- 172.21.15.111:0/2388830147 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f13fc1a1500 con 0x7f13fc109e90 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 msgr2=0x7f13e003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 0x7f13e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13e8006fd0 tx=0x7f13e8006e40 comp rx=0 tx=0).stop 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 msgr2=0x7f13fc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13ec011850 tx=0x7f13ec011c10 comp rx=0 tx=0).stop 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 shutdown_connections 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f13e0039050 0x7f13e003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 --2- 172.21.15.111:0/2388830147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13fc109e90 0x7f13fc19de70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 >> 172.21.15.111:0/2388830147 conn(0x7f13fc07d260 msgr2=0x7f13fc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 shutdown_connections 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:42.673+0000 7f13fb200700 1 -- 172.21.15.111:0/2388830147 wait complete. 2024-04-24T22:30:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling cephadm module... 2024-04-24T22:30:44.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setuser ceph since I am not root 2024-04-24T22:30:44.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setgroup ceph since I am not root 2024-04-24T22:30:44.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1019270049' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T22:30:44.836 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:44.583+0000 7f0db2d42200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:30:44.836 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:44.834+0000 7f0db2d42200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.965+0000 7f6970c00700 1 Processor -- start 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.965+0000 7f6970c00700 1 -- start start 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.966+0000 7f6970c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.966+0000 7f6970c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f696c07be80 con 0x7f696c108880 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.966+0000 7f6969e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.966+0000 7f6969e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34558/0 (socket says 172.21.15.111:34558) 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.966+0000 7f6969e00700 1 -- 172.21.15.111:0/3964875058 learned_addr learned my addr 172.21.15.111:0/3964875058 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.967+0000 7f6969e00700 1 -- 172.21.15.111:0/3964875058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c10b1b0 con 0x7f696c108880 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.967+0000 7f6969e00700 1 --2- 172.21.15.111:0/3964875058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f695c0098a0 tx=0x7f695c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a125bdfee528b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.967+0000 7f6968a00700 1 -- 172.21.15.111:0/3964875058 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f695c004160 con 0x7f696c108880 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6968a00700 1 -- 172.21.15.111:0/3964875058 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f695c004300 con 0x7f696c108880 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6968a00700 1 -- 172.21.15.111:0/3964875058 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f695c011b50 con 0x7f696c108880 2024-04-24T22:30:45.015 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6970c00700 1 -- 172.21.15.111:0/3964875058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 msgr2=0x7f696c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6970c00700 1 --2- 172.21.15.111:0/3964875058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f695c0098a0 tx=0x7f695c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6970c00700 1 -- 172.21.15.111:0/3964875058 shutdown_connections 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6970c00700 1 --2- 172.21.15.111:0/3964875058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c10ac70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.968+0000 7f6970c00700 1 -- 172.21.15.111:0/3964875058 >> 172.21.15.111:0/3964875058 conn(0x7f696c1022d0 msgr2=0x7f696c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.969+0000 7f6970c00700 1 -- 172.21.15.111:0/3964875058 shutdown_connections 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.969+0000 7f6970c00700 1 -- 172.21.15.111:0/3964875058 wait complete. 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6970c00700 1 Processor -- start 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6970c00700 1 -- start start 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6970c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6970c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f696c10c1e0 con 0x7f696c108880 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6969e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6969e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34568/0 (socket says 172.21.15.111:34568) 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.970+0000 7f6969e00700 1 -- 172.21.15.111:0/1019270049 learned_addr learned my addr 172.21.15.111:0/1019270049 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:45.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6969e00700 1 -- 172.21.15.111:0/1019270049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c108640 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6969e00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f695c011b60 tx=0x7f695c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f695c003970 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f695c003b10 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f695c024610 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.971+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c106830 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.972+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c106d70 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.973+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f695c01a9f0 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.973+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c063890 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.973+0000 7f6962a00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 0x7f695003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.973+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f695c011e00 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.973+0000 7f6969400700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 0x7f695003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.974+0000 7f6969400700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 0x7f695003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:43.978+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f695c016260 con 0x7f696c108880 2024-04-24T22:30:45.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.257+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f696c106ff0 con 0x7f696c108880 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.311+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f695c019e10 con 0x7f696c108880 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6962a00700 1 -- 172.21.15.111:0/1019270049 <== mon.0 v2:172.21.15.111:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f695c024c40 con 0x7f696c108880 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 msgr2=0x7f695003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 0x7f695003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).stop 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 msgr2=0x7f696c108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f695c011b60 tx=0x7f695c011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 shutdown_connections 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f6950039050 0x7f695003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 --2- 172.21.15.111:0/1019270049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f696c108880 0x7f696c108100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.314+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 >> 172.21.15.111:0/1019270049 conn(0x7f696c1022d0 msgr2=0x7f696c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.315+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 shutdown_connections 2024-04-24T22:30:45.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:44.315+0000 7f6970c00700 1 -- 172.21.15.111:0/1019270049 wait complete. 2024-04-24T22:30:45.154 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:44.966+0000 7f0db2d42200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:30:45.406 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:45 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1019270049' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T22:30:45.406 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:45 smithi111 ceph-mon[27746]: mgrmap e5: x(active, since 5s) 2024-04-24T22:30:45.406 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:45 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:45.381+0000 7f0db2d42200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:30:45.669 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:45 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:45.498+0000 7f0db2d42200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:30:46.578 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:46 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/857424414' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T22:30:46.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:46.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.667+0000 7f9fb3200700 1 Processor -- start 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.667+0000 7f9fb3200700 1 -- start start 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.668+0000 7f9fb3200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.668+0000 7f9fb3200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac108770 con 0x7f9fac107e10 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.668+0000 7f9fabe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.668+0000 7f9fabe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48920/0 (socket says 172.21.15.111:48920) 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.668+0000 7f9fabe00700 1 -- 172.21.15.111:0/3785896685 learned_addr learned my addr 172.21.15.111:0/3785896685 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.669+0000 7f9fabe00700 1 -- 172.21.15.111:0/3785896685 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac108f70 con 0x7f9fac107e10 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.669+0000 7f9fabe00700 1 --2- 172.21.15.111:0/3785896685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0098a0 tx=0x7f9f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd94672e3db56e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.669+0000 7f9fab400700 1 -- 172.21.15.111:0/3785896685 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c004160 con 0x7f9fac107e10 2024-04-24T22:30:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.669+0000 7f9fab400700 1 -- 172.21.15.111:0/3785896685 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f9c004300 con 0x7f9fac107e10 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.669+0000 7f9fab400700 1 -- 172.21.15.111:0/3785896685 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c011b50 con 0x7f9fac107e10 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 -- 172.21.15.111:0/3785896685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 msgr2=0x7f9fac108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 --2- 172.21.15.111:0/3785896685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0098a0 tx=0x7f9f9c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 -- 172.21.15.111:0/3785896685 shutdown_connections 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 --2- 172.21.15.111:0/3785896685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac108230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 -- 172.21.15.111:0/3785896685 >> 172.21.15.111:0/3785896685 conn(0x7f9fac103410 msgr2=0x7f9fac105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 -- 172.21.15.111:0/3785896685 shutdown_connections 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.670+0000 7f9fb3200700 1 -- 172.21.15.111:0/3785896685 wait complete. 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.671+0000 7f9fb3200700 1 Processor -- start 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fb3200700 1 -- start start 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fb3200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fb3200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10c3b0 con 0x7f9fac107e10 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fabe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:46.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fabe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48926/0 (socket says 172.21.15.111:48926) 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.672+0000 7f9fabe00700 1 -- 172.21.15.111:0/857424414 learned_addr learned my addr 172.21.15.111:0/857424414 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fabe00700 1 -- 172.21.15.111:0/857424414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac07e080 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fabe00700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c011b60 tx=0x7f9f9c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c003970 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f9c003b10 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac07c270 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c024610 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.673+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac07c7b0 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.674+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9f9c024910 con 0x7f9fac107e10 2024-04-24T22:30:46.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.674+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f88005360 con 0x7f9fac107e10 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.675+0000 7f9fa9600700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 0x7f9f9003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.675+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f9c011e00 con 0x7f9fac107e10 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.675+0000 7f9fa3400700 1 -- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 msgr2=0x7f9f9003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.675+0000 7f9fa3400700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 0x7f9f9003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.680+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f9c0172b0 con 0x7f9fac107e10 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.876+0000 7f9fa3400700 1 -- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 msgr2=0x7f9f9003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.876+0000 7f9fa3400700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 0x7f9f9003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.950+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9f88005e60 con 0x7f9fac107e10 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.951+0000 7f9fa9600700 1 -- 172.21.15.111:0/857424414 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9f9c00ba00 con 0x7f9fac107e10 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.953+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 msgr2=0x7f9f9003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.953+0000 7f9fb3200700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 0x7f9f9003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.953+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 msgr2=0x7f9fac07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.953+0000 7f9fb3200700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c011b60 tx=0x7f9f9c011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:46.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 shutdown_connections 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7f9f90039050 0x7f9f9003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 --2- 172.21.15.111:0/857424414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fac107e10 0x7f9fac07db40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 >> 172.21.15.111:0/857424414 conn(0x7f9fac103410 msgr2=0x7f9fac103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 shutdown_connections 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:45.954+0000 7f9fb3200700 1 -- 172.21.15.111:0/857424414 wait complete. 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for the mgr to restart... 2024-04-24T22:30:46.599 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr epoch 5... 2024-04-24T22:30:47.171 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:47.057+0000 7f0db2d42200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:30:47.501 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:47.288+0000 7f0db2d42200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:30:47.782 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:47.498+0000 7f0db2d42200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:30:48.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:47.871+0000 7f0db2d42200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:30:48.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:47.991+0000 7f0db2d42200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:30:48.494 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:48 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:48.241+0000 7f0db2d42200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:30:48.495 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:48 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:48.492+0000 7f0db2d42200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:30:49.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:49.268+0000 7f0db2d42200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:30:50.185 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:49.809+0000 7f0db2d42200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:30:50.185 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:49.933+0000 7f0db2d42200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:30:50.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:50.192+0000 7f0db2d42200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:30:51.260 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:50.827+0000 7f0db2d42200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:30:51.260 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:50.947+0000 7f0db2d42200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:30:51.261 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:51.257+0000 7f0db2d42200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:30:51.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:51.433+0000 7f0db2d42200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:30:51.829 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:51.555+0000 7f0db2d42200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:30:51.829 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:51.705+0000 7f0db2d42200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:30:51.829 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:51.827+0000 7f0db2d42200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:30:53.107 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:52 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:52.827+0000 7f0db2d42200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:30:53.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:30:53.104+0000 7f0db2d42200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: Active manager daemon x restarted 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: Activating manager daemon x 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: mgrmap e6: x(active, starting, since 0.0095896s) 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: Manager daemon x is now available 2024-04-24T22:30:53.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:53 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:30:54.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: Found migration_current of "None". Setting to last migration. 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:30:54.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:54 smithi111 ceph-mon[27746]: mgrmap e7: x(active, since 1.01521s) 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.169+0000 7fb506c00700 1 Processor -- start 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.169+0000 7fb506c00700 1 -- start start 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.170+0000 7fb506c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.170+0000 7fb506c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb50007c8a0 con 0x7fb50007de20 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.170+0000 7fb505800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.170+0000 7fb505800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48932/0 (socket says 172.21.15.111:48932) 2024-04-24T22:30:55.028 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.170+0000 7fb505800700 1 -- 172.21.15.111:0/127016680 learned_addr learned my addr 172.21.15.111:0/127016680 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.171+0000 7fb505800700 1 -- 172.21.15.111:0/127016680 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50007ca20 con 0x7fb50007de20 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.171+0000 7fb505800700 1 --2- 172.21.15.111:0/127016680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098a0 tx=0x7fb4f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67f920b689f138 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.171+0000 7fb4ffe00700 1 -- 172.21.15.111:0/127016680 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f4004160 con 0x7fb50007de20 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.172+0000 7fb4ffe00700 1 -- 172.21.15.111:0/127016680 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4f4004300 con 0x7fb50007de20 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.172+0000 7fb4ffe00700 1 -- 172.21.15.111:0/127016680 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f4011b50 con 0x7fb50007de20 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.172+0000 7fb506c00700 1 -- 172.21.15.111:0/127016680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 msgr2=0x7fb50007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.172+0000 7fb506c00700 1 --2- 172.21.15.111:0/127016680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098a0 tx=0x7fb4f4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.173+0000 7fb506c00700 1 -- 172.21.15.111:0/127016680 shutdown_connections 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.173+0000 7fb506c00700 1 --2- 172.21.15.111:0/127016680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50007c2d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.173+0000 7fb506c00700 1 -- 172.21.15.111:0/127016680 >> 172.21.15.111:0/127016680 conn(0x7fb500103210 msgr2=0x7fb500105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.173+0000 7fb506c00700 1 -- 172.21.15.111:0/127016680 shutdown_connections 2024-04-24T22:30:55.029 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.173+0000 7fb506c00700 1 -- 172.21.15.111:0/127016680 wait complete. 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.174+0000 7fb506c00700 1 Processor -- start 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.174+0000 7fb506c00700 1 -- start start 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb506c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb506c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010e200 con 0x7fb50007de20 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb505800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb505800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48944/0 (socket says 172.21.15.111:48944) 2024-04-24T22:30:55.030 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb505800700 1 -- 172.21.15.111:0/3945075485 learned_addr learned my addr 172.21.15.111:0/3945075485 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.175+0000 7fb505800700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50019e350 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb505800700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4011b60 tx=0x7fb4f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f4003970 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4f4003b10 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f4024610 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50019e5e0 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.176+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5001a12d0 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.177+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb4f40248c0 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.178+0000 7fb4fe000700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.178+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb4e803bc70 con 0x7fb4e80390a0 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.178+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4f4011e00 con 0x7fb50007de20 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.178+0000 7fb504e00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.178+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:30:55.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.379+0000 7fb504e00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.379+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.780+0000 7fb504e00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:47.780+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:48.581+0000 7fb504e00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:48.581+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:50.183+0000 7fb504e00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2142544018 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:50.183+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:53.114+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fb4f4024bb0 con 0x7fb50007de20 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:53.115+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 msgr2=0x7fb4e803b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:53.115+0000 7fb4fe000700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.117+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fb4f4029cf0 con 0x7fb50007de20 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.118+0000 7fb4fe000700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 0x7fb4e803edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.118+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb4f4021e70 con 0x7fb4e803c9d0 2024-04-24T22:30:55.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.118+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 0x7fb4e803edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.119+0000 7fb504e00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 0x7fb4e803edc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec003a10 tx=0x7fb4ec0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.119+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb4f4021e70 con 0x7fb4e803c9d0 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.127+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb500063890 con 0x7fb4e803c9d0 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.127+0000 7fb4fe000700 1 -- 172.21.15.111:0/3945075485 <== mgr.14120 v2:172.21.15.111:6800/3447332391 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb500063890 con 0x7fb4e803c9d0 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 msgr2=0x7fb4e803edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 0x7fb4e803edc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec003a10 tx=0x7fb4ec0092b0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 msgr2=0x7fb50019de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4011b60 tx=0x7fb4f4011c40 comp rx=0 tx=0).stop 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 shutdown_connections 2024-04-24T22:30:55.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fb4e803c9d0 0x7fb4e803edc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb50007de20 0x7fb50019de10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 --2- 172.21.15.111:0/3945075485 >> [v2:172.21.15.111:6800/2142544018,v1:172.21.15.111:6801/2142544018] conn(0x7fb4e80390a0 0x7fb4e803b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 >> 172.21.15.111:0/3945075485 conn(0x7fb500103210 msgr2=0x7fb500103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 shutdown_connections 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:54.128+0000 7fb506c00700 1 -- 172.21.15.111:0/3945075485 wait complete. 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:mgr epoch 5 is available 2024-04-24T22:30:55.034 INFO:teuthology.orchestra.run.smithi111.stdout:Setting orchestrator backend to cephadm... 2024-04-24T22:30:55.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: [24/Apr/2024:22:30:54] ENGINE Bus STARTING 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: [24/Apr/2024:22:30:54] ENGINE Serving on https://172.21.15.111:7150 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: [24/Apr/2024:22:30:54] ENGINE Serving on http://172.21.15.111:8765 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: [24/Apr/2024:22:30:54] ENGINE Bus STARTED 2024-04-24T22:30:55.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:55 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:30:56.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:56 smithi111 ceph-mon[27746]: mgrmap e8: x(active, since 2s) 2024-04-24T22:30:56.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:56 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:30:56.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:56 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:30:56.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.624+0000 7f4bb9000700 1 Processor -- start 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.624+0000 7f4bb9000700 1 -- start start 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.625+0000 7f4bb9000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.625+0000 7f4bb9000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb407c8c0 con 0x7f4bb407de90 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.625+0000 7f4bb1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.626+0000 7f4bb1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42366/0 (socket says 172.21.15.111:42366) 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.626+0000 7f4bb1e00700 1 -- 172.21.15.111:0/3883971926 learned_addr learned my addr 172.21.15.111:0/3883971926 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.626+0000 7f4bb1e00700 1 -- 172.21.15.111:0/3883971926 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb407ca40 con 0x7f4bb407de90 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.626+0000 7f4bb1e00700 1 --2- 172.21.15.111:0/3883971926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098a0 tx=0x7f4ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caab8439da414725 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:56.594 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.627+0000 7f4bb0a00700 1 -- 172.21.15.111:0/3883971926 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4004160 con 0x7f4bb407de90 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.627+0000 7f4bb0a00700 1 -- 172.21.15.111:0/3883971926 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ba4004300 con 0x7f4bb407de90 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.627+0000 7f4bb0a00700 1 -- 172.21.15.111:0/3883971926 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4011b50 con 0x7f4bb407de90 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.627+0000 7f4bb9000700 1 -- 172.21.15.111:0/3883971926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 msgr2=0x7f4bb407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.627+0000 7f4bb9000700 1 --2- 172.21.15.111:0/3883971926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098a0 tx=0x7f4ba4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.628+0000 7f4bb9000700 1 -- 172.21.15.111:0/3883971926 shutdown_connections 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.628+0000 7f4bb9000700 1 --2- 172.21.15.111:0/3883971926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb407c2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.628+0000 7f4bb9000700 1 -- 172.21.15.111:0/3883971926 >> 172.21.15.111:0/3883971926 conn(0x7f4bb4103410 msgr2=0x7f4bb4105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.628+0000 7f4bb9000700 1 -- 172.21.15.111:0/3883971926 shutdown_connections 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.628+0000 7f4bb9000700 1 -- 172.21.15.111:0/3883971926 wait complete. 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.629+0000 7f4bb9000700 1 Processor -- start 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.629+0000 7f4bb9000700 1 -- start start 2024-04-24T22:30:56.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb9000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb9000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb410e400 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42380/0 (socket says 172.21.15.111:42380) 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb1e00700 1 -- 172.21.15.111:0/590295561 learned_addr learned my addr 172.21.15.111:0/590295561 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.630+0000 7f4bb1e00700 1 -- 172.21.15.111:0/590295561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb419e570 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4bb1e00700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4011850 tx=0x7f4ba4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4003af0 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ba4020070 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb419e800 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba40195a0 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.631+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41a14f0 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.632+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4ba4019850 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.632+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb4063890 con 0x7f4bb407de90 2024-04-24T22:30:56.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.633+0000 7f4baaa00700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 0x7f4b9803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.633+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ba4011e00 con 0x7f4bb407de90 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.633+0000 7f4bb1400700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 0x7f4b9803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.634+0000 7f4bb1400700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 0x7f4b9803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0006fd0 tx=0x7f4ba0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.639+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ba4003c90 con 0x7f4bb407de90 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.864+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4bb41046e0 con 0x7f4b98039100 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.877+0000 7f4baaa00700 1 -- 172.21.15.111:0/590295561 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4bb41046e0 con 0x7f4b98039100 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 msgr2=0x7f4b9803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 0x7f4b9803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0006fd0 tx=0x7f4ba0006e40 comp rx=0 tx=0).stop 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 msgr2=0x7f4bb419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4011850 tx=0x7f4ba4011c10 comp rx=0 tx=0).stop 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 shutdown_connections 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f4b98039100 0x7f4b9803b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 --2- 172.21.15.111:0/590295561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb407de90 0x7f4bb419e030 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 >> 172.21.15.111:0/590295561 conn(0x7f4bb4103410 msgr2=0x7f4bb4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:56.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.880+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 shutdown_connections 2024-04-24T22:30:56.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:55.881+0000 7f4bb9000700 1 -- 172.21.15.111:0/590295561 wait complete. 2024-04-24T22:30:57.210 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:57 smithi111 ceph-mon[27746]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:30:58.232 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.249+0000 7f08d8a00700 1 Processor -- start 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d8a00700 1 -- start start 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d8a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d8a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08d410a880 con 0x7f08d4109e90 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:58.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42388/0 (socket says 172.21.15.111:42388) 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.250+0000 7f08d3400700 1 -- 172.21.15.111:0/3241250447 learned_addr learned my addr 172.21.15.111:0/3241250447 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.251+0000 7f08d3400700 1 -- 172.21.15.111:0/3241250447 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d410aff0 con 0x7f08d4109e90 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.251+0000 7f08d3400700 1 --2- 172.21.15.111:0/3241250447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08c40098a0 tx=0x7f08c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3486d7782f3dab83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.251+0000 7f08d2000700 1 -- 172.21.15.111:0/3241250447 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c4004160 con 0x7f08d4109e90 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.251+0000 7f08d2000700 1 -- 172.21.15.111:0/3241250447 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08c4004300 con 0x7f08d4109e90 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.251+0000 7f08d2000700 1 -- 172.21.15.111:0/3241250447 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c4011b50 con 0x7f08d4109e90 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 -- 172.21.15.111:0/3241250447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 msgr2=0x7f08d410a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 --2- 172.21.15.111:0/3241250447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08c40098a0 tx=0x7f08c4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 -- 172.21.15.111:0/3241250447 shutdown_connections 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 --2- 172.21.15.111:0/3241250447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d410a2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 -- 172.21.15.111:0/3241250447 >> 172.21.15.111:0/3241250447 conn(0x7f08d407d260 msgr2=0x7f08d407d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 -- 172.21.15.111:0/3241250447 shutdown_connections 2024-04-24T22:30:58.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.252+0000 7f08d8a00700 1 -- 172.21.15.111:0/3241250447 wait complete. 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.253+0000 7f08d8a00700 1 Processor -- start 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d8a00700 1 -- start start 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d8a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d8a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08d410e3a0 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42394/0 (socket says 172.21.15.111:42394) 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d3400700 1 -- 172.21.15.111:0/1592908800 learned_addr learned my addr 172.21.15.111:0/1592908800 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.254+0000 7f08d3400700 1 -- 172.21.15.111:0/1592908800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d419e3b0 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08d3400700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08c4011850 tx=0x7f08c4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c4003af0 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08c4020070 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c40195a0 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d419e640 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.255+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d41a1330 con 0x7f08d4109e90 2024-04-24T22:30:58.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.256+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f08c4019850 con 0x7f08d4109e90 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.256+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4063890 con 0x7f08d4109e90 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.257+0000 7f08cbe00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 0x7f08b803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.257+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08c4011e00 con 0x7f08d4109e90 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.257+0000 7f08d2a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 0x7f08b803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.258+0000 7f08d2a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 0x7f08b803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006fd0 tx=0x7f08c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.263+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08c400e260 con 0x7f08d4109e90 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.485+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f08d4107c50 con 0x7f08b80390a0 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.487+0000 7f08cbe00700 1 -- 172.21.15.111:0/1592908800 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f08d4107c50 con 0x7f08b80390a0 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.489+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 msgr2=0x7f08b803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.489+0000 7f08d8a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 0x7f08b803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006fd0 tx=0x7f08c0006e40 comp rx=0 tx=0).stop 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 msgr2=0x7f08d419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08c4011850 tx=0x7f08c4011c10 comp rx=0 tx=0).stop 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 shutdown_connections 2024-04-24T22:30:58.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f08b80390a0 0x7f08b803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:58.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 --2- 172.21.15.111:0/1592908800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d4109e90 0x7f08d419de70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:30:58.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 >> 172.21.15.111:0/1592908800 conn(0x7f08d407d260 msgr2=0x7f08d4107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:30:58.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 shutdown_connections 2024-04-24T22:30:58.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:57.490+0000 7f08d8a00700 1 -- 172.21.15.111:0/1592908800 wait complete. 2024-04-24T22:30:58.241 INFO:teuthology.orchestra.run.smithi111.stdout:Generating ssh key... 2024-04-24T22:30:58.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:30:58 smithi111 ceph-mon[27746]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:30:59.353 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: Generating public/private rsa key pair. 2024-04-24T22:30:59.353 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: Your identification has been saved in /tmp/tmp8dko8osx/key. 2024-04-24T22:30:59.353 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: Your public key has been saved in /tmp/tmp8dko8osx/key.pub. 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: The key fingerprint is: 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: SHA256:MJwR6yWuKsqKWrtW6b9Cidv2/XxaSXhCU635XaTn6z8 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: The key's randomart image is: 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: +---[RSA 3072]----+ 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | o. .. | 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | . + . . .| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | B . o o o | 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | o = . oo . o| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | . oo S o o. +.| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | . =. + .. o| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: | .*. o .| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: |+.oo= . . .. E | 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: |O++o ++...+o ..+| 2024-04-24T22:30:59.354 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:30:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: +----[SHA256]-----+ 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.861+0000 7fab57200700 1 Processor -- start 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.861+0000 7fab57200700 1 -- start start 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.861+0000 7fab57200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.861+0000 7fab57200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab5807c8a0 con 0x7fab5807de20 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.862+0000 7fab55e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.862+0000 7fab55e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42410/0 (socket says 172.21.15.111:42410) 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.862+0000 7fab55e00700 1 -- 172.21.15.111:0/2747942229 learned_addr learned my addr 172.21.15.111:0/2747942229 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:00.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.862+0000 7fab55e00700 1 -- 172.21.15.111:0/2747942229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5807ca20 con 0x7fab5807de20 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.863+0000 7fab55e00700 1 --2- 172.21.15.111:0/2747942229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab480098a0 tx=0x7fab48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5336197abf1abd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.863+0000 7fab54a00700 1 -- 172.21.15.111:0/2747942229 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab48004160 con 0x7fab5807de20 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.863+0000 7fab54a00700 1 -- 172.21.15.111:0/2747942229 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab48004300 con 0x7fab5807de20 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.863+0000 7fab54a00700 1 -- 172.21.15.111:0/2747942229 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab48011b50 con 0x7fab5807de20 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 -- 172.21.15.111:0/2747942229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 msgr2=0x7fab5807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 --2- 172.21.15.111:0/2747942229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab480098a0 tx=0x7fab48009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 -- 172.21.15.111:0/2747942229 shutdown_connections 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 --2- 172.21.15.111:0/2747942229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5807c2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 -- 172.21.15.111:0/2747942229 >> 172.21.15.111:0/2747942229 conn(0x7fab58103210 msgr2=0x7fab58105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 -- 172.21.15.111:0/2747942229 shutdown_connections 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.864+0000 7fab57200700 1 -- 172.21.15.111:0/2747942229 wait complete. 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.865+0000 7fab57200700 1 Processor -- start 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.865+0000 7fab57200700 1 -- start start 2024-04-24T22:31:00.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.865+0000 7fab57200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.865+0000 7fab57200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab5810e200 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.866+0000 7fab55e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.866+0000 7fab55e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42412/0 (socket says 172.21.15.111:42412) 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.866+0000 7fab55e00700 1 -- 172.21.15.111:0/107517666 learned_addr learned my addr 172.21.15.111:0/107517666 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.866+0000 7fab55e00700 1 -- 172.21.15.111:0/107517666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5819e360 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.866+0000 7fab55e00700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab48011b60 tx=0x7fab48011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.867+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab48003970 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.867+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab48003b10 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.867+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab48024610 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.867+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5819e5f0 con 0x7fab5807de20 2024-04-24T22:31:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.867+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a12e0 con 0x7fab5807de20 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.868+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fab4801a9f0 con 0x7fab5807de20 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.868+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab58063890 con 0x7fab5807de20 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.868+0000 7fab4ea00700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 0x7fab3c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.868+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab48011e00 con 0x7fab5807de20 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.869+0000 7fab55400700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 0x7fab3c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.870+0000 7fab55400700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 0x7fab3c03b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:58.874+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab4800b3e0 con 0x7fab5807de20 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.088+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fab58104540 con 0x7fab3c0390a0 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.330+0000 7fab4ea00700 1 -- 172.21.15.111:0/107517666 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab58104540 con 0x7fab3c0390a0 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.333+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 msgr2=0x7fab3c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.333+0000 7fab57200700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 0x7fab3c03b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.333+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 msgr2=0x7fab5819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.333+0000 7fab57200700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab48011b60 tx=0x7fab48011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:00.037 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 shutdown_connections 2024-04-24T22:31:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fab3c0390a0 0x7fab3c03b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 --2- 172.21.15.111:0/107517666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5807de20 0x7fab5819de20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 >> 172.21.15.111:0/107517666 conn(0x7fab58103210 msgr2=0x7fab58103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 shutdown_connections 2024-04-24T22:31:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:30:59.334+0000 7fab57200700 1 -- 172.21.15.111:0/107517666 wait complete. 2024-04-24T22:31:00.590 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:00 smithi111 ceph-mon[27746]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:00.590 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:00 smithi111 ceph-mon[27746]: Generating ssh key... 2024-04-24T22:31:00.590 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:00 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:00.590 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:00 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:01.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:01 smithi111 ceph-mon[27746]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:01.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:31:01.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.650+0000 7f20a6e00700 1 Processor -- start 2024-04-24T22:31:01.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.650+0000 7f20a6e00700 1 -- start start 2024-04-24T22:31:01.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.651+0000 7f20a6e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.651+0000 7f20a6e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010aa00 con 0x7f20a010a010 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.651+0000 7f209fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.651+0000 7f209fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42422/0 (socket says 172.21.15.111:42422) 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.651+0000 7f209fe00700 1 -- 172.21.15.111:0/2889082802 learned_addr learned my addr 172.21.15.111:0/2889082802 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.652+0000 7f209fe00700 1 -- 172.21.15.111:0/2889082802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a010b1c0 con 0x7f20a010a010 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.652+0000 7f209fe00700 1 --2- 172.21.15.111:0/2889082802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b424243510fc5c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.652+0000 7f209ea00700 1 -- 172.21.15.111:0/2889082802 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2090004160 con 0x7f20a010a010 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.652+0000 7f209ea00700 1 -- 172.21.15.111:0/2889082802 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2090004300 con 0x7f20a010a010 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.652+0000 7f209ea00700 1 -- 172.21.15.111:0/2889082802 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2090011b50 con 0x7f20a010a010 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 -- 172.21.15.111:0/2889082802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 msgr2=0x7f20a010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 --2- 172.21.15.111:0/2889082802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 -- 172.21.15.111:0/2889082802 shutdown_connections 2024-04-24T22:31:01.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 --2- 172.21.15.111:0/2889082802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a010a430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 -- 172.21.15.111:0/2889082802 >> 172.21.15.111:0/2889082802 conn(0x7f20a007d3a0 msgr2=0x7f20a007d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 -- 172.21.15.111:0/2889082802 shutdown_connections 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.653+0000 7f20a6e00700 1 -- 172.21.15.111:0/2889082802 wait complete. 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.654+0000 7f20a6e00700 1 Processor -- start 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.654+0000 7f20a6e00700 1 -- start start 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f20a6e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f20a6e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010e570 con 0x7f20a010a010 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f209fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f209fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42436/0 (socket says 172.21.15.111:42436) 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f209fe00700 1 -- 172.21.15.111:0/304257293 learned_addr learned my addr 172.21.15.111:0/304257293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.655+0000 7f209fe00700 1 -- 172.21.15.111:0/304257293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a019e560 con 0x7f20a010a010 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f209fe00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2090011850 tx=0x7f2090011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2090003af0 con 0x7f20a010a010 2024-04-24T22:31:01.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2090020070 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a019e7f0 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20900195a0 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.656+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01a14e0 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.657+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a0063890 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.657+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f209001a430 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.658+0000 7f209cc00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 0x7f208403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.658+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2090011e00 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.658+0000 7f209f400700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 0x7f208403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.659+0000 7f209f400700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 0x7f208403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f208c006fd0 tx=0x7f208c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.663+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f209000e260 con 0x7f20a010a010 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.887+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f20a0107cc0 con 0x7f20840390a0 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.888+0000 7f209cc00700 1 -- 172.21.15.111:0/304257293 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f20a0107cc0 con 0x7f20840390a0 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 msgr2=0x7f208403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:01.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 0x7f208403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f208c006fd0 tx=0x7f208c006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 msgr2=0x7f20a019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2090011850 tx=0x7f2090011c10 comp rx=0 tx=0).stop 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 shutdown_connections 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f20840390a0 0x7f208403b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 --2- 172.21.15.111:0/304257293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a010a010 0x7f20a019e020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 >> 172.21.15.111:0/304257293 conn(0x7f20a007d3a0 msgr2=0x7f20a01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 shutdown_connections 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:00.891+0000 7f20a6e00700 1 -- 172.21.15.111:0/304257293 wait complete. 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T22:31:01.603 INFO:teuthology.orchestra.run.smithi111.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T22:31:01.610 INFO:teuthology.orchestra.run.smithi111.stdout:Adding host smithi111... 2024-04-24T22:31:03.739 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:03 smithi111 ceph-mon[27746]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "addr": "172.21.15.111", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:04.573 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:04 smithi111 ceph-mon[27746]: Deploying cephadm binary to smithi111 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Added host 'smithi111' with addr '172.21.15.111' 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.202+0000 7f9599c00700 1 Processor -- start 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.203+0000 7f9599c00700 1 -- start start 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.203+0000 7f9599c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.203+0000 7f9599c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9594082460 con 0x7f959410eca0 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.204+0000 7f9593e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:09.038 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.204+0000 7f9593e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42452/0 (socket says 172.21.15.111:42452) 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.204+0000 7f9593e00700 1 -- 172.21.15.111:0/3350640451 learned_addr learned my addr 172.21.15.111:0/3350640451 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.205+0000 7f9593e00700 1 -- 172.21.15.111:0/3350640451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95941115d0 con 0x7f959410eca0 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.205+0000 7f9593e00700 1 --2- 172.21.15.111:0/3350640451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95840098a0 tx=0x7f9584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85e18eff1f8cfae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.205+0000 7f9592a00700 1 -- 172.21.15.111:0/3350640451 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584004160 con 0x7f959410eca0 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.205+0000 7f9592a00700 1 -- 172.21.15.111:0/3350640451 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9584004300 con 0x7f959410eca0 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.205+0000 7f9592a00700 1 -- 172.21.15.111:0/3350640451 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584011b50 con 0x7f959410eca0 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 -- 172.21.15.111:0/3350640451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 msgr2=0x7f9594111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 --2- 172.21.15.111:0/3350640451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95840098a0 tx=0x7f9584009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 -- 172.21.15.111:0/3350640451 shutdown_connections 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 --2- 172.21.15.111:0/3350640451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f9594111090 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:09.039 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 -- 172.21.15.111:0/3350640451 >> 172.21.15.111:0/3350640451 conn(0x7f959407d260 msgr2=0x7f959407f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 -- 172.21.15.111:0/3350640451 shutdown_connections 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.206+0000 7f9599c00700 1 -- 172.21.15.111:0/3350640451 wait complete. 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.207+0000 7f9599c00700 1 Processor -- start 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.207+0000 7f9599c00700 1 -- start start 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9599c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9599c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95941125f0 con 0x7f959410eca0 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9593e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9593e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42464/0 (socket says 172.21.15.111:42464) 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9593e00700 1 -- 172.21.15.111:0/3109301398 learned_addr learned my addr 172.21.15.111:0/3109301398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.208+0000 7f9593e00700 1 -- 172.21.15.111:0/3109301398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95941a2730 con 0x7f959410eca0 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9593e00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9584011b60 tx=0x7f9584011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584003970 con 0x7f959410eca0 2024-04-24T22:31:09.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9584003b10 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95941a29c0 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584024610 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.209+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941a56b0 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.210+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f958401a9f0 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.210+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9594063890 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.211+0000 7f9590c00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 0x7f957803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.211+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9584011e00 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.211+0000 7f9593400700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 0x7f957803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.212+0000 7f9593400700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 0x7f957803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9580006fd0 tx=0x7f9580006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.217+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f958400b3e0 con 0x7f959410eca0 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:02.442+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "addr": "172.21.15.111", "target": ["mon-mgr", ""]}) -- 0x7f959407e570 con 0x7f9578039050 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.303+0000 7f9590c00700 1 -- 172.21.15.111:0/3109301398 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f959407e570 con 0x7f9578039050 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 msgr2=0x7f957803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:09.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 0x7f957803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9580006fd0 tx=0x7f9580006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 msgr2=0x7f95941a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9584011b60 tx=0x7f9584011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 shutdown_connections 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f9578039050 0x7f957803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 --2- 172.21.15.111:0/3109301398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f959410eca0 0x7f95941a21f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.307+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 >> 172.21.15.111:0/3109301398 conn(0x7f959407d260 msgr2=0x7f959407de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.308+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 shutdown_connections 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:08.308+0000 7f9599c00700 1 -- 172.21.15.111:0/3109301398 wait complete. 2024-04-24T22:31:09.042 INFO:teuthology.orchestra.run.smithi111.stdout:Deploying unmanaged mon service... 2024-04-24T22:31:09.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:09 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:09.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:09 smithi111 ceph-mon[27746]: Added host smithi111 2024-04-24T22:31:09.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:09 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:11.080 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T22:31:11.080 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.891+0000 7fa838a00700 1 Processor -- start 2024-04-24T22:31:11.080 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.891+0000 7fa838a00700 1 -- start start 2024-04-24T22:31:11.080 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.891+0000 7fa838a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.891+0000 7fa838a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa834108640 con 0x7fa834107c50 2024-04-24T22:31:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.892+0000 7fa832800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.892+0000 7fa832800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54728/0 (socket says 172.21.15.111:54728) 2024-04-24T22:31:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.892+0000 7fa832800700 1 -- 172.21.15.111:0/3767027039 learned_addr learned my addr 172.21.15.111:0/3767027039 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:11.082 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.892+0000 7fa832800700 1 -- 172.21.15.111:0/3767027039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa834108e00 con 0x7fa834107c50 2024-04-24T22:31:11.082 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.892+0000 7fa832800700 1 --2- 172.21.15.111:0/3767027039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8240098a0 tx=0x7fa824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5002e6809cce8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:11.082 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa831400700 1 -- 172.21.15.111:0/3767027039 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa824004160 con 0x7fa834107c50 2024-04-24T22:31:11.082 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa831400700 1 -- 172.21.15.111:0/3767027039 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa824004300 con 0x7fa834107c50 2024-04-24T22:31:11.083 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa831400700 1 -- 172.21.15.111:0/3767027039 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa824011b50 con 0x7fa834107c50 2024-04-24T22:31:11.083 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa838a00700 1 -- 172.21.15.111:0/3767027039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 msgr2=0x7fa834108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:11.083 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa838a00700 1 --2- 172.21.15.111:0/3767027039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8240098a0 tx=0x7fa824009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:11.083 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa838a00700 1 -- 172.21.15.111:0/3767027039 shutdown_connections 2024-04-24T22:31:11.084 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.893+0000 7fa838a00700 1 --2- 172.21.15.111:0/3767027039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa834108070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.894+0000 7fa838a00700 1 -- 172.21.15.111:0/3767027039 >> 172.21.15.111:0/3767027039 conn(0x7fa834103210 msgr2=0x7fa834105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.894+0000 7fa838a00700 1 -- 172.21.15.111:0/3767027039 shutdown_connections 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.894+0000 7fa838a00700 1 -- 172.21.15.111:0/3767027039 wait complete. 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.895+0000 7fa838a00700 1 Processor -- start 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.895+0000 7fa838a00700 1 -- start start 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.895+0000 7fa838a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.895+0000 7fa838a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa83410c1b0 con 0x7fa834107c50 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.895+0000 7fa832800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.896+0000 7fa832800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54734/0 (socket says 172.21.15.111:54734) 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.896+0000 7fa832800700 1 -- 172.21.15.111:0/449550921 learned_addr learned my addr 172.21.15.111:0/449550921 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.896+0000 7fa832800700 1 -- 172.21.15.111:0/449550921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83419e3c0 con 0x7fa834107c50 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.896+0000 7fa832800700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa824009fb0 tx=0x7fa824011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.896+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8240038b0 con 0x7fa834107c50 2024-04-24T22:31:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.897+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa824011e50 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.897+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa824019550 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.897+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83419e650 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.897+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8341a1340 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.898+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa834063890 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.898+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa824003a50 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.898+0000 7fa82b400700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 0x7fa81803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.899+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa82404d390 con 0x7fa834107c50 2024-04-24T22:31:11.088 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.899+0000 7fa831e00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 0x7fa81803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.899+0000 7fa831e00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 0x7fa81803b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:09.904+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa824020240 con 0x7fa834107c50 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.157+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa834104430 con 0x7fa818039000 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.167+0000 7fa82b400700 1 -- 172.21.15.111:0/449550921 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa834104430 con 0x7fa818039000 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.169+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 msgr2=0x7fa81803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.169+0000 7fa838a00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 0x7fa81803b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.169+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 msgr2=0x7fa83419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.169+0000 7fa838a00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa824009fb0 tx=0x7fa824011bf0 comp rx=0 tx=0).stop 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 shutdown_connections 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fa818039000 0x7fa81803b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 --2- 172.21.15.111:0/449550921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa834107c50 0x7fa83419de80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 >> 172.21.15.111:0/449550921 conn(0x7fa834103210 msgr2=0x7fa834103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 shutdown_connections 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:10.170+0000 7fa838a00700 1 -- 172.21.15.111:0/449550921 wait complete. 2024-04-24T22:31:11.089 INFO:teuthology.orchestra.run.smithi111.stdout:Deploying unmanaged mgr service... 2024-04-24T22:31:11.254 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:11 smithi111 ceph-mon[27746]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:11.255 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:11 smithi111 ceph-mon[27746]: Saving service mon spec with placement count:5 2024-04-24T22:31:11.255 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:11 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:13.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:12 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:13.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:12 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:13.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:12 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:13.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T22:31:13.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.455+0000 7f2a8fa00700 1 Processor -- start 2024-04-24T22:31:13.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.455+0000 7f2a8fa00700 1 -- start start 2024-04-24T22:31:13.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.455+0000 7f2a8fa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.455+0000 7f2a8fa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8807c880 con 0x7f2a8807de50 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.456+0000 7f2a8d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.456+0000 7f2a8d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54736/0 (socket says 172.21.15.111:54736) 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.456+0000 7f2a8d000700 1 -- 172.21.15.111:0/4293732726 learned_addr learned my addr 172.21.15.111:0/4293732726 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.456+0000 7f2a8d000700 1 -- 172.21.15.111:0/4293732726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8807ca00 con 0x7f2a8807de50 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.457+0000 7f2a8d000700 1 --2- 172.21.15.111:0/4293732726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0098a0 tx=0x7f2a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c988b5af36990f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.457+0000 7f2a87400700 1 -- 172.21.15.111:0/4293732726 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c004160 con 0x7f2a8807de50 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.457+0000 7f2a87400700 1 -- 172.21.15.111:0/4293732726 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c004300 con 0x7f2a8807de50 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.457+0000 7f2a87400700 1 -- 172.21.15.111:0/4293732726 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c011b20 con 0x7f2a8807de50 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 -- 172.21.15.111:0/4293732726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 msgr2=0x7f2a8807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:13.448 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/4293732726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0098a0 tx=0x7f2a7c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 -- 172.21.15.111:0/4293732726 shutdown_connections 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/4293732726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8807c2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 -- 172.21.15.111:0/4293732726 >> 172.21.15.111:0/4293732726 conn(0x7f2a88103410 msgr2=0x7f2a88105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 -- 172.21.15.111:0/4293732726 shutdown_connections 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.458+0000 7f2a8fa00700 1 -- 172.21.15.111:0/4293732726 wait complete. 2024-04-24T22:31:13.453 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.459+0000 7f2a8fa00700 1 Processor -- start 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.459+0000 7f2a8fa00700 1 -- start start 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.459+0000 7f2a8fa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.459+0000 7f2a8fa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8810e490 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.460+0000 7f2a8d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.460+0000 7f2a8d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54742/0 (socket says 172.21.15.111:54742) 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.460+0000 7f2a8d000700 1 -- 172.21.15.111:0/1343753477 learned_addr learned my addr 172.21.15.111:0/1343753477 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.460+0000 7f2a8d000700 1 -- 172.21.15.111:0/1343753477 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8819e4d0 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.460+0000 7f2a8d000700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c011b60 tx=0x7f2a7c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.461+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c003b60 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.461+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c01a910 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.461+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c024930 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.461+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8819e760 con 0x7f2a8807de50 2024-04-24T22:31:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.461+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8819ebc0 con 0x7f2a8807de50 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.462+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2a7c022020 con 0x7f2a8807de50 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.462+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a88063890 con 0x7f2a8807de50 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.463+0000 7f2a85600700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 0x7f2a7003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.463+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a7c04d310 con 0x7f2a8807de50 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.463+0000 7f2a87e00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 0x7f2a7003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.464+0000 7f2a87e00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 0x7f2a7003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006fd0 tx=0x7f2a74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.473+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2a7c00b3e0 con 0x7f2a8807de50 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.711+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2a881045f0 con 0x7f2a70039050 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.718+0000 7f2a85600700 1 -- 172.21.15.111:0/1343753477 <== mgr.14120 v2:172.21.15.111:6800/3447332391 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a881045f0 con 0x7f2a70039050 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.720+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 msgr2=0x7f2a7003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.720+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 0x7f2a7003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006fd0 tx=0x7f2a74006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.720+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 msgr2=0x7f2a8819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.720+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c011b60 tx=0x7f2a7c011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 shutdown_connections 2024-04-24T22:31:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2a70039050 0x7f2a7003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:13.456 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 --2- 172.21.15.111:0/1343753477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8807de50 0x7f2a8819df90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:13.456 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 >> 172.21.15.111:0/1343753477 conn(0x7f2a88103410 msgr2=0x7f2a88103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:13.456 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 shutdown_connections 2024-04-24T22:31:13.456 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:12.721+0000 7f2a8fa00700 1 -- 172.21.15.111:0/1343753477 wait complete. 2024-04-24T22:31:14.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:14 smithi111 ceph-mon[27746]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:14.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:14 smithi111 ceph-mon[27746]: Saving service mgr spec with placement count:2 2024-04-24T22:31:14.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:14 smithi111 ceph-mon[27746]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:15.489 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.103+0000 7fc92c400700 1 Processor -- start 2024-04-24T22:31:15.489 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.103+0000 7fc92c400700 1 -- start start 2024-04-24T22:31:15.489 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.103+0000 7fc92c400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:15.489 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.103+0000 7fc92c400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410aa50 con 0x7fc92410a060 2024-04-24T22:31:15.489 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.104+0000 7fc929a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.104+0000 7fc929a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54746/0 (socket says 172.21.15.111:54746) 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.104+0000 7fc929a00700 1 -- 172.21.15.111:0/73198038 learned_addr learned my addr 172.21.15.111:0/73198038 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.104+0000 7fc929a00700 1 -- 172.21.15.111:0/73198038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92410b210 con 0x7fc92410a060 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.105+0000 7fc929a00700 1 --2- 172.21.15.111:0/73198038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9180098a0 tx=0x7fc918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb2486d709bb783 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.105+0000 7fc923e00700 1 -- 172.21.15.111:0/73198038 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc918004160 con 0x7fc92410a060 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.105+0000 7fc923e00700 1 -- 172.21.15.111:0/73198038 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918004300 con 0x7fc92410a060 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.105+0000 7fc923e00700 1 -- 172.21.15.111:0/73198038 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc918011b20 con 0x7fc92410a060 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 -- 172.21.15.111:0/73198038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 msgr2=0x7fc92410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 --2- 172.21.15.111:0/73198038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9180098a0 tx=0x7fc918009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 -- 172.21.15.111:0/73198038 shutdown_connections 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 --2- 172.21.15.111:0/73198038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92410a480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 -- 172.21.15.111:0/73198038 >> 172.21.15.111:0/73198038 conn(0x7fc92407d600 msgr2=0x7fc92407da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 -- 172.21.15.111:0/73198038 shutdown_connections 2024-04-24T22:31:15.490 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.106+0000 7fc92c400700 1 -- 172.21.15.111:0/73198038 wait complete. 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.108+0000 7fc92c400700 1 Processor -- start 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.108+0000 7fc92c400700 1 -- start start 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.108+0000 7fc92c400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.108+0000 7fc92c400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410e5c0 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.109+0000 7fc929a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.109+0000 7fc929a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54750/0 (socket says 172.21.15.111:54750) 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.109+0000 7fc929a00700 1 -- 172.21.15.111:0/909924286 learned_addr learned my addr 172.21.15.111:0/909924286 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.109+0000 7fc929a00700 1 -- 172.21.15.111:0/909924286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92419e4b0 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.109+0000 7fc929a00700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9180117e0 tx=0x7fc918011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.110+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc918003bf0 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.110+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91801a910 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.110+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9180249a0 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.110+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92419e740 con 0x7fc92410a060 2024-04-24T22:31:15.491 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.110+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92419eba0 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.111+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc918022020 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.111+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc924063890 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.112+0000 7fc922000700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 0x7fc90c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.112+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc91804d440 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.112+0000 7fc929000700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 0x7fc90c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.113+0000 7fc929000700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 0x7fc90c03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc910006fd0 tx=0x7fc910006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.118+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc918011e20 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.353+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc9241a1980 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.361+0000 7fc922000700 1 -- 172.21.15.111:0/909924286 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc918004300 con 0x7fc92410a060 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 msgr2=0x7fc90c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 0x7fc90c03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc910006fd0 tx=0x7fc910006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 msgr2=0x7fc92419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9180117e0 tx=0x7fc918011c10 comp rx=0 tx=0).stop 2024-04-24T22:31:15.492 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 shutdown_connections 2024-04-24T22:31:15.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc90c039050 0x7fc90c03b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:15.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 --2- 172.21.15.111:0/909924286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc92410a060 0x7fc92419df70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:15.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 >> 172.21.15.111:0/909924286 conn(0x7fc92407d600 msgr2=0x7fc924107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:15.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 shutdown_connections 2024-04-24T22:31:15.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:14.365+0000 7fc92c400700 1 -- 172.21.15.111:0/909924286 wait complete. 2024-04-24T22:31:15.640 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:15 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/909924286' entity='client.admin' 2024-04-24T22:31:16.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:16 smithi111 ceph-mon[27746]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:16 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:16 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3189941247' entity='client.admin' 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.105+0000 7f5822000700 1 Processor -- start 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.105+0000 7f5822000700 1 -- start start 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.105+0000 7f5822000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.105+0000 7f5822000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f581c07c880 con 0x7f581c07de50 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.106+0000 7f581b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.106+0000 7f581b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60644/0 (socket says 172.21.15.111:60644) 2024-04-24T22:31:17.016 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.106+0000 7f581b400700 1 -- 172.21.15.111:0/1227500049 learned_addr learned my addr 172.21.15.111:0/1227500049 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.107+0000 7f581b400700 1 -- 172.21.15.111:0/1227500049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c07ca00 con 0x7f581c07de50 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.107+0000 7f581b400700 1 --2- 172.21.15.111:0/1227500049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f580c0098a0 tx=0x7f580c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28ffe3b2ce11af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.107+0000 7f581a000700 1 -- 172.21.15.111:0/1227500049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f580c004160 con 0x7f581c07de50 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.107+0000 7f581a000700 1 -- 172.21.15.111:0/1227500049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c004300 con 0x7f581c07de50 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.107+0000 7f581a000700 1 -- 172.21.15.111:0/1227500049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f580c011b20 con 0x7f581c07de50 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 -- 172.21.15.111:0/1227500049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 msgr2=0x7f581c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 --2- 172.21.15.111:0/1227500049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f580c0098a0 tx=0x7f580c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 -- 172.21.15.111:0/1227500049 shutdown_connections 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 --2- 172.21.15.111:0/1227500049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c07c2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 -- 172.21.15.111:0/1227500049 >> 172.21.15.111:0/1227500049 conn(0x7f581c103410 msgr2=0x7f581c105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 -- 172.21.15.111:0/1227500049 shutdown_connections 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.108+0000 7f5822000700 1 -- 172.21.15.111:0/1227500049 wait complete. 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.109+0000 7f5822000700 1 Processor -- start 2024-04-24T22:31:17.017 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f5822000700 1 -- start start 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f5822000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f5822000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f581c10e490 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f581b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f581b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60648/0 (socket says 172.21.15.111:60648) 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.110+0000 7f581b400700 1 -- 172.21.15.111:0/3189941247 learned_addr learned my addr 172.21.15.111:0/3189941247 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f581b400700 1 -- 172.21.15.111:0/3189941247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c19e4d0 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f581b400700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f580c011b60 tx=0x7f580c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f580c003b60 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c01a910 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f580c024930 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.111+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c19e760 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.112+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c19ebc0 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.113+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c063890 con 0x7f581c07de50 2024-04-24T22:31:17.018 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.113+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f580c022020 con 0x7f581c07de50 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.113+0000 7f5813e00700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 0x7f580003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.113+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f580c04d1d0 con 0x7f581c07de50 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.113+0000 7f581aa00700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 0x7f580003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.114+0000 7f581aa00700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 0x7f580003b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5808006fd0 tx=0x7f5808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.119+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f580c01a4e0 con 0x7f581c07de50 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.343+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f581c1a1650 con 0x7f581c07de50 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.347+0000 7f5813e00700 1 -- 172.21.15.111:0/3189941247 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f580c02d3f0 con 0x7f581c07de50 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 msgr2=0x7f580003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 0x7f580003b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5808006fd0 tx=0x7f5808006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 msgr2=0x7f581c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f580c011b60 tx=0x7f580c011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 shutdown_connections 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f58000390a0 0x7f580003b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 --2- 172.21.15.111:0/3189941247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c07de50 0x7f581c19df90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:17.019 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 >> 172.21.15.111:0/3189941247 conn(0x7f581c103410 msgr2=0x7f581c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:17.020 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.350+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 shutdown_connections 2024-04-24T22:31:17.020 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:16.351+0000 7f5822000700 1 -- 172.21.15.111:0/3189941247 wait complete. 2024-04-24T22:31:17.020 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling the dashboard module... 2024-04-24T22:31:18.630 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-mon[27746]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:18.630 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-mon[27746]: from='mgr.14120 172.21.15.111:0/4206554653' entity='mgr.x' 2024-04-24T22:31:18.630 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3294697445' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T22:31:18.889 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setuser ceph since I am not root 2024-04-24T22:31:18.889 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setgroup ceph since I am not root 2024-04-24T22:31:18.889 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:18.876+0000 7f34fa9f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:31:19.249 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:19.117+0000 7f34fa9f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:31:19.250 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:19.247+0000 7f34fa9f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:31:19.770 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:19.656+0000 7f34fa9f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:31:19.771 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3294697445' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T22:31:19.771 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-mon[27746]: mgrmap e9: x(active, since 25s) 2024-04-24T22:31:20.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:19.768+0000 7f34fa9f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.630+0000 7fc1f9000700 1 Processor -- start 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.631+0000 7fc1f9000700 1 -- start start 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.631+0000 7fc1f9000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.631+0000 7fc1f9000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f407c8a0 con 0x7fc1f407de20 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.631+0000 7fc1f1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.631+0000 7fc1f1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60664/0 (socket says 172.21.15.111:60664) 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.632+0000 7fc1f1e00700 1 -- 172.21.15.111:0/3693573818 learned_addr learned my addr 172.21.15.111:0/3693573818 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:20.157 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.632+0000 7fc1f1e00700 1 -- 172.21.15.111:0/3693573818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f407ca20 con 0x7fc1f407de20 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.632+0000 7fc1f1e00700 1 --2- 172.21.15.111:0/3693573818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc97dac37abfa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.633+0000 7fc1f0a00700 1 -- 172.21.15.111:0/3693573818 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4004160 con 0x7fc1f407de20 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.633+0000 7fc1f0a00700 1 -- 172.21.15.111:0/3693573818 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e4004300 con 0x7fc1f407de20 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.633+0000 7fc1f0a00700 1 -- 172.21.15.111:0/3693573818 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4011b20 con 0x7fc1f407de20 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.633+0000 7fc1f9000700 1 -- 172.21.15.111:0/3693573818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 msgr2=0x7fc1f407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.633+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3693573818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.634+0000 7fc1f9000700 1 -- 172.21.15.111:0/3693573818 shutdown_connections 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.634+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3693573818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f407c2d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.634+0000 7fc1f9000700 1 -- 172.21.15.111:0/3693573818 >> 172.21.15.111:0/3693573818 conn(0x7fc1f41033f0 msgr2=0x7fc1f4105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.634+0000 7fc1f9000700 1 -- 172.21.15.111:0/3693573818 shutdown_connections 2024-04-24T22:31:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.634+0000 7fc1f9000700 1 -- 172.21.15.111:0/3693573818 wait complete. 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.635+0000 7fc1f9000700 1 Processor -- start 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.635+0000 7fc1f9000700 1 -- start start 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f9000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f9000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f410e430 con 0x7fc1f407de20 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60666/0 (socket says 172.21.15.111:60666) 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f1e00700 1 -- 172.21.15.111:0/3294697445 learned_addr learned my addr 172.21.15.111:0/3294697445 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.636+0000 7fc1f1e00700 1 -- 172.21.15.111:0/3294697445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f419e4e0 con 0x7fc1f407de20 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1f1e00700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40116a0 tx=0x7fc1e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4003a40 con 0x7fc1f407de20 2024-04-24T22:31:20.159 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e4003be0 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1e401a750 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f419e770 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.637+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f419ebd0 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.638+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc1e401aa00 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.639+0000 7fc1eaa00700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 0x7fc1d803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.639+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc1e404d6c0 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.639+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f4063890 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.639+0000 7fc1f1400700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 0x7fc1d803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.640+0000 7fc1f1400700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 0x7fc1d803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006fd0 tx=0x7fc1e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.645+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc1e4011e20 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:17.940+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc1f41a1950 con 0x7fc1f407de20 2024-04-24T22:31:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.562+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc1e404d010 con 0x7fc1f407de20 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.565+0000 7fc1eaa00700 1 -- 172.21.15.111:0/3294697445 <== mon.0 v2:172.21.15.111:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc1e402d3f0 con 0x7fc1f407de20 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.567+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 msgr2=0x7fc1d803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.567+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 0x7fc1d803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006fd0 tx=0x7fc1e0006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.567+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 msgr2=0x7fc1f419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.567+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40116a0 tx=0x7fc1e4011ba0 comp rx=0 tx=0).stop 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 shutdown_connections 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7fc1d8039050 0x7fc1d803b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 --2- 172.21.15.111:0/3294697445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f407de20 0x7fc1f419dfa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 >> 172.21.15.111:0/3294697445 conn(0x7fc1f41033f0 msgr2=0x7fc1f4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 shutdown_connections 2024-04-24T22:31:20.161 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:18.568+0000 7fc1f9000700 1 -- 172.21.15.111:0/3294697445 wait complete. 2024-04-24T22:31:21.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:21 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/201023613' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T22:31:21.724 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:21.337+0000 7f34fa9f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:31:21.724 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:21.563+0000 7f34fa9f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T22:31:21.743 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.730+0000 7f2735e00700 1 Processor -- start 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.730+0000 7f2735e00700 1 -- start start 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.731+0000 7f2735e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.731+0000 7f2735e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f273007bda0 con 0x7f2730108960 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.731+0000 7f272ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.731+0000 7f272ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60696/0 (socket says 172.21.15.111:60696) 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.731+0000 7f272ea00700 1 -- 172.21.15.111:0/3814691178 learned_addr learned my addr 172.21.15.111:0/3814691178 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.732+0000 7f272ea00700 1 -- 172.21.15.111:0/3814691178 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273010b290 con 0x7f2730108960 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.732+0000 7f272ea00700 1 --2- 172.21.15.111:0/3814691178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27200098a0 tx=0x7f2720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed3636714f913b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.732+0000 7f272d600700 1 -- 172.21.15.111:0/3814691178 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2720004160 con 0x7f2730108960 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.732+0000 7f272d600700 1 -- 172.21.15.111:0/3814691178 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2720004300 con 0x7f2730108960 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.732+0000 7f272d600700 1 -- 172.21.15.111:0/3814691178 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2720011b20 con 0x7f2730108960 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 -- 172.21.15.111:0/3814691178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 msgr2=0x7f273010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:21.744 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 --2- 172.21.15.111:0/3814691178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27200098a0 tx=0x7f2720009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 -- 172.21.15.111:0/3814691178 shutdown_connections 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 --2- 172.21.15.111:0/3814691178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273010ad50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 -- 172.21.15.111:0/3814691178 >> 172.21.15.111:0/3814691178 conn(0x7f27301022d0 msgr2=0x7f27301046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 -- 172.21.15.111:0/3814691178 shutdown_connections 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.733+0000 7f2735e00700 1 -- 172.21.15.111:0/3814691178 wait complete. 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.734+0000 7f2735e00700 1 Processor -- start 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.734+0000 7f2735e00700 1 -- start start 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f2735e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f2735e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f273010c230 con 0x7f2730108960 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f272ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f272ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60702/0 (socket says 172.21.15.111:60702) 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f272ea00700 1 -- 172.21.15.111:0/201023613 learned_addr learned my addr 172.21.15.111:0/201023613 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.735+0000 7f272ea00700 1 -- 172.21.15.111:0/201023613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273019c2f0 con 0x7f2730108960 2024-04-24T22:31:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f272ea00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2720011b60 tx=0x7f2720011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2720003b60 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272001a910 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273019c580 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2720024930 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.736+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273019c9e0 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.737+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2720022020 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.737+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2730063890 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.738+0000 7f2734a00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 0x7f271403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.738+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f272004d350 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.738+0000 7f272e000700 1 -- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 msgr2=0x7f271403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.738+0000 7f272e000700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 0x7f271403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.747+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f272000fdd0 con 0x7f2730108960 2024-04-24T22:31:21.746 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.939+0000 7f272e000700 1 -- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 msgr2=0x7f271403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:20.939+0000 7f272e000700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 0x7f271403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.028+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f273019f390 con 0x7f2730108960 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.029+0000 7f2734a00700 1 -- 172.21.15.111:0/201023613 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f272004cca0 con 0x7f2730108960 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 msgr2=0x7f271403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 0x7f271403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 msgr2=0x7f273019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2720011b60 tx=0x7f2720011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 shutdown_connections 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f2714039100 0x7f271403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 --2- 172.21.15.111:0/201023613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730108960 0x7f273019bdb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 >> 172.21.15.111:0/201023613 conn(0x7f27301022d0 msgr2=0x7f2730102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 shutdown_connections 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:21.033+0000 7f2735e00700 1 -- 172.21.15.111:0/201023613 wait complete. 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for the mgr to restart... 2024-04-24T22:31:21.747 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr epoch 9... 2024-04-24T22:31:22.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:21.780+0000 7f34fa9f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:31:22.296 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:22.167+0000 7f34fa9f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:31:22.296 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:22.294+0000 7f34fa9f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:31:22.937 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:22.550+0000 7f34fa9f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:31:22.937 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:22.799+0000 7f34fa9f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:31:24.039 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:23 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:23.567+0000 7f34fa9f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:31:24.505 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:24.116+0000 7f34fa9f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:31:24.505 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:24.241+0000 7f34fa9f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:31:25.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:24.502+0000 7f34fa9f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:31:25.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:25.137+0000 7f34fa9f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:31:25.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:25.258+0000 7f34fa9f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:31:25.875 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:25.573+0000 7f34fa9f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:31:25.875 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:25.749+0000 7f34fa9f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:31:26.149 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:25.873+0000 7f34fa9f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:31:26.150 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:26.025+0000 7f34fa9f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:31:26.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:26.147+0000 7f34fa9f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:31:27.424 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:27.133+0000 7f34fa9f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:31:27.783 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:31:27.421+0000 7f34fa9f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:31:27.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: Active manager daemon x restarted 2024-04-24T22:31:27.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: Activating manager daemon x 2024-04-24T22:31:27.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: mgrmap e10: x(active, starting, since 0.00615219s) 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:31:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:27 smithi111 ceph-mon[27746]: Manager daemon x is now available 2024-04-24T22:31:28.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:28.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:31:28.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:31:28.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:28 smithi111 ceph-mon[27746]: mgrmap e11: x(active, since 1.00991s) 2024-04-24T22:31:29.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: [24/Apr/2024:22:31:27] ENGINE Bus STARTING 2024-04-24T22:31:29.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: [24/Apr/2024:22:31:27] ENGINE Serving on https://172.21.15.111:7150 2024-04-24T22:31:29.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: [24/Apr/2024:22:31:28] ENGINE Serving on http://172.21.15.111:8765 2024-04-24T22:31:29.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: [24/Apr/2024:22:31:28] ENGINE Bus STARTED 2024-04-24T22:31:29.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T22:31:29.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:29 smithi111 ceph-mon[27746]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.325+0000 7f1b3cc00700 1 Processor -- start 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.325+0000 7f1b3cc00700 1 -- start start 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.325+0000 7f1b3cc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.325+0000 7f1b3cc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3807be80 con 0x7f1b381086c0 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.326+0000 7f1b37400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.326+0000 7f1b37400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60714/0 (socket says 172.21.15.111:60714) 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.326+0000 7f1b37400700 1 -- 172.21.15.111:0/3494793059 learned_addr learned my addr 172.21.15.111:0/3494793059 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:29.756 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.326+0000 7f1b37400700 1 -- 172.21.15.111:0/3494793059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3810aff0 con 0x7f1b381086c0 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.327+0000 7f1b37400700 1 --2- 172.21.15.111:0/3494793059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b280098a0 tx=0x7f1b28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155ef842c67e63f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.327+0000 7f1b36000700 1 -- 172.21.15.111:0/3494793059 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28004160 con 0x7f1b381086c0 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.327+0000 7f1b36000700 1 -- 172.21.15.111:0/3494793059 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b28004300 con 0x7f1b381086c0 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.327+0000 7f1b36000700 1 -- 172.21.15.111:0/3494793059 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28011b20 con 0x7f1b381086c0 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 -- 172.21.15.111:0/3494793059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 msgr2=0x7f1b3810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/3494793059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b280098a0 tx=0x7f1b28009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 -- 172.21.15.111:0/3494793059 shutdown_connections 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/3494793059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3810aab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 -- 172.21.15.111:0/3494793059 >> 172.21.15.111:0/3494793059 conn(0x7f1b381020d0 msgr2=0x7f1b38104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 -- 172.21.15.111:0/3494793059 shutdown_connections 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.328+0000 7f1b3cc00700 1 -- 172.21.15.111:0/3494793059 wait complete. 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.329+0000 7f1b3cc00700 1 Processor -- start 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.329+0000 7f1b3cc00700 1 -- start start 2024-04-24T22:31:29.757 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b3cc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b3cc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3810c060 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b37400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b37400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60722/0 (socket says 172.21.15.111:60722) 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b37400700 1 -- 172.21.15.111:0/2804223218 learned_addr learned my addr 172.21.15.111:0/2804223218 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.330+0000 7f1b37400700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3819c130 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b37400700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b28011b60 tx=0x7f1b28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28003b60 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2801a910 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28024930 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3819c3c0 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.331+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3819c820 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.332+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1b28022020 con 0x7f1b381086c0 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.333+0000 7f1b2fe00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:29.758 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.333+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b1c03bcd0 con 0x7f1b1c039100 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.333+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b2804d330 con 0x7f1b381086c0 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.333+0000 7f1b36a00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.333+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.534+0000 7f1b36a00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.534+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.935+0000 7f1b36a00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:22.935+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:23.736+0000 7f1b36a00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:23.736+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:31:29.759 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:25.338+0000 7f1b36a00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3447332391 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:25.338+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:27.430+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f1b2802d3f0 con 0x7f1b381086c0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:27.430+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 msgr2=0x7f1b1c03b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:27.430+0000 7f1b2fe00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.432+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f1b2804f110 con 0x7f1b381086c0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.432+0000 7f1b2fe00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 0x7f1b1c03efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.432+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b28006880 con 0x7f1b1c03cbc0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.435+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 0x7f1b1c03efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.435+0000 7f1b36a00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 0x7f1b1c03efb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b24003a10 tx=0x7f1b24009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.437+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f1b28006880 con 0x7f1b1c03cbc0 2024-04-24T22:31:29.760 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.444+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1b38063890 con 0x7f1b1c03cbc0 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.444+0000 7f1b2fe00700 1 -- 172.21.15.111:0/2804223218 <== mgr.14152 v2:172.21.15.111:6800/1872264645 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1b38063890 con 0x7f1b1c03cbc0 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 msgr2=0x7f1b1c03efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 0x7f1b1c03efb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b24003a10 tx=0x7f1b24009270 comp rx=0 tx=0).stop 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 msgr2=0x7f1b3819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b28011b60 tx=0x7f1b28011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 shutdown_connections 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1b1c03cbc0 0x7f1b1c03efb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b381086c0 0x7f1b3819bbf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 --2- 172.21.15.111:0/2804223218 >> [v2:172.21.15.111:6800/3447332391,v1:172.21.15.111:6801/3447332391] conn(0x7f1b1c039100 0x7f1b1c03b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 >> 172.21.15.111:0/2804223218 conn(0x7f1b381020d0 msgr2=0x7f1b38102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 shutdown_connections 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:28.445+0000 7f1b3cc00700 1 -- 172.21.15.111:0/2804223218 wait complete. 2024-04-24T22:31:29.761 INFO:teuthology.orchestra.run.smithi111.stdout:mgr epoch 9 is available 2024-04-24T22:31:29.762 INFO:teuthology.orchestra.run.smithi111.stdout:Generating a dashboard self-signed certificate... 2024-04-24T22:31:30.766 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:30 smithi111 ceph-mon[27746]: mgrmap e12: x(active, since 2s) 2024-04-24T22:31:30.767 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:31.620 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T22:31:31.620 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.519+0000 7fa54f200700 1 Processor -- start 2024-04-24T22:31:31.620 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.519+0000 7fa54f200700 1 -- start start 2024-04-24T22:31:31.620 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.519+0000 7fa54f200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.519+0000 7fa54f200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010a7d0 con 0x7fa550109e70 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.520+0000 7fa54de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.520+0000 7fa54de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55122/0 (socket says 172.21.15.111:55122) 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.520+0000 7fa54de00700 1 -- 172.21.15.111:0/3403265689 learned_addr learned my addr 172.21.15.111:0/3403265689 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.520+0000 7fa54de00700 1 -- 172.21.15.111:0/3403265689 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55010af80 con 0x7fa550109e70 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.520+0000 7fa54de00700 1 --2- 172.21.15.111:0/3403265689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa5400098a0 tx=0x7fa540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4df5bb7d4831608 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.521+0000 7fa54ca00700 1 -- 172.21.15.111:0/3403265689 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540004160 con 0x7fa550109e70 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.521+0000 7fa54ca00700 1 -- 172.21.15.111:0/3403265689 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa540004300 con 0x7fa550109e70 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.521+0000 7fa54ca00700 1 -- 172.21.15.111:0/3403265689 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540011b20 con 0x7fa550109e70 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.521+0000 7fa54f200700 1 -- 172.21.15.111:0/3403265689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 msgr2=0x7fa55010a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.521+0000 7fa54f200700 1 --2- 172.21.15.111:0/3403265689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa5400098a0 tx=0x7fa540009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.522+0000 7fa54f200700 1 -- 172.21.15.111:0/3403265689 shutdown_connections 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.522+0000 7fa54f200700 1 --2- 172.21.15.111:0/3403265689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55010a290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.522+0000 7fa54f200700 1 -- 172.21.15.111:0/3403265689 >> 172.21.15.111:0/3403265689 conn(0x7fa55007d230 msgr2=0x7fa55007d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.522+0000 7fa54f200700 1 -- 172.21.15.111:0/3403265689 shutdown_connections 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.522+0000 7fa54f200700 1 -- 172.21.15.111:0/3403265689 wait complete. 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.523+0000 7fa54f200700 1 Processor -- start 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.523+0000 7fa54f200700 1 -- start start 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.523+0000 7fa54f200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.523+0000 7fa54f200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010e3c0 con 0x7fa550109e70 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.524+0000 7fa54de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.524+0000 7fa54de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55134/0 (socket says 172.21.15.111:55134) 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.524+0000 7fa54de00700 1 -- 172.21.15.111:0/253546777 learned_addr learned my addr 172.21.15.111:0/253546777 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.524+0000 7fa54de00700 1 -- 172.21.15.111:0/253546777 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55019e320 con 0x7fa550109e70 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.524+0000 7fa54de00700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa540011b60 tx=0x7fa540011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.525+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540003b60 con 0x7fa550109e70 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.525+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54001a910 con 0x7fa550109e70 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.525+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55019e5b0 con 0x7fa550109e70 2024-04-24T22:31:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.525+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540024930 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.525+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55019ea10 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.526+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa540022020 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.526+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa550063890 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.527+0000 7fa546a00700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 0x7fa53403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.527+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa54004d440 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.527+0000 7fa54d400700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 0x7fa53403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.528+0000 7fa54d400700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 0x7fa53403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006fd0 tx=0x7fa53c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.532+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa540024c50 con 0x7fa550109e70 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.764+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa550107b40 con 0x7fa5340390a0 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.843+0000 7fa546a00700 1 -- 172.21.15.111:0/253546777 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa550107b40 con 0x7fa5340390a0 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.846+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 msgr2=0x7fa53403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.846+0000 7fa54f200700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 0x7fa53403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006fd0 tx=0x7fa53c006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.846+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 msgr2=0x7fa55019dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.846+0000 7fa54f200700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa540011b60 tx=0x7fa540011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 shutdown_connections 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa5340390a0 0x7fa53403b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 --2- 172.21.15.111:0/253546777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa550109e70 0x7fa55019dde0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 >> 172.21.15.111:0/253546777 conn(0x7fa55007d230 msgr2=0x7fa550107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 shutdown_connections 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:30.847+0000 7fa54f200700 1 -- 172.21.15.111:0/253546777 wait complete. 2024-04-24T22:31:31.624 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial admin user... 2024-04-24T22:31:31.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:31 smithi111 ceph-mon[27746]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:31.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:31.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:31.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:33.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:32 smithi111 ceph-mon[27746]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:33.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:33.435 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Fo/keNusTY/ReVJmzK0xGOpUCHlFxDMHI9wdS.5WnuW6hjOy4jcXu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713997892, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T22:31:33.435 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.234+0000 7fa3ea600700 1 Processor -- start 2024-04-24T22:31:33.435 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.234+0000 7fa3ea600700 1 -- start start 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.234+0000 7fa3ea600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.234+0000 7fa3ea600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4106ad0 con 0x7fa3e41060e0 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.235+0000 7fa3e3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.235+0000 7fa3e3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55146/0 (socket says 172.21.15.111:55146) 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.235+0000 7fa3e3400700 1 -- 172.21.15.111:0/1780930009 learned_addr learned my addr 172.21.15.111:0/1780930009 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:33.436 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.236+0000 7fa3e3400700 1 -- 172.21.15.111:0/1780930009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e4109070 con 0x7fa3e41060e0 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.236+0000 7fa3e3400700 1 --2- 172.21.15.111:0/1780930009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40098a0 tx=0x7fa3d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ac8234f87fc99c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.236+0000 7fa3e2000700 1 -- 172.21.15.111:0/1780930009 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d4004160 con 0x7fa3e41060e0 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.236+0000 7fa3e2000700 1 -- 172.21.15.111:0/1780930009 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d4004300 con 0x7fa3e41060e0 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.236+0000 7fa3e2000700 1 -- 172.21.15.111:0/1780930009 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d4011b20 con 0x7fa3e41060e0 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 -- 172.21.15.111:0/1780930009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 msgr2=0x7fa3e4106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 --2- 172.21.15.111:0/1780930009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40098a0 tx=0x7fa3d4009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:33.438 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 -- 172.21.15.111:0/1780930009 shutdown_connections 2024-04-24T22:31:33.438 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 --2- 172.21.15.111:0/1780930009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e4106500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:33.438 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 -- 172.21.15.111:0/1780930009 >> 172.21.15.111:0/1780930009 conn(0x7fa3e4101a30 msgr2=0x7fa3e4103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:33.438 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 -- 172.21.15.111:0/1780930009 shutdown_connections 2024-04-24T22:31:33.438 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.237+0000 7fa3ea600700 1 -- 172.21.15.111:0/1780930009 wait complete. 2024-04-24T22:31:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3ea600700 1 Processor -- start 2024-04-24T22:31:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3ea600700 1 -- start start 2024-04-24T22:31:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3ea600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3ea600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e410c420 con 0x7fa3e41060e0 2024-04-24T22:31:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3e3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3e3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55152/0 (socket says 172.21.15.111:55152) 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.239+0000 7fa3e3400700 1 -- 172.21.15.111:0/3749587584 learned_addr learned my addr 172.21.15.111:0/3749587584 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3e3400700 1 -- 172.21.15.111:0/3749587584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e4108e30 con 0x7fa3e41060e0 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3e3400700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d4003b60 con 0x7fa3e41060e0 2024-04-24T22:31:33.440 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e4107020 con 0x7fa3e41060e0 2024-04-24T22:31:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d401a910 con 0x7fa3e41060e0 2024-04-24T22:31:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.240+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d4024930 con 0x7fa3e41060e0 2024-04-24T22:31:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.241+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e41074e0 con 0x7fa3e41060e0 2024-04-24T22:31:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.242+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa3d4022020 con 0x7fa3e41060e0 2024-04-24T22:31:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.242+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e4063890 con 0x7fa3e41060e0 2024-04-24T22:31:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.242+0000 7fa3dbe00700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 0x7fa3c803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.242+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3d404d550 con 0x7fa3e41060e0 2024-04-24T22:31:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.243+0000 7fa3e2a00700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 0x7fa3c803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.243+0000 7fa3e2a00700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 0x7fa3c803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006fd0 tx=0x7fa3d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.248+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3d402dc40 con 0x7fa3e41060e0 2024-04-24T22:31:33.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.497+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa3e4107760 con 0x7fa3c8039100 2024-04-24T22:31:33.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.737+0000 7fa3dbe00700 1 -- 172.21.15.111:0/3749587584 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa3e4107760 con 0x7fa3c8039100 2024-04-24T22:31:33.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 msgr2=0x7fa3c803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:33.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 0x7fa3c803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006fd0 tx=0x7fa3d0006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:33.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 msgr2=0x7fa3e41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d4011c10 comp rx=0 tx=0).stop 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 shutdown_connections 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3c8039100 0x7fa3c803b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 --2- 172.21.15.111:0/3749587584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3e41060e0 0x7fa3e41088f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 >> 172.21.15.111:0/3749587584 conn(0x7fa3e4101a30 msgr2=0x7fa3e4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.740+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 shutdown_connections 2024-04-24T22:31:33.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:32.741+0000 7fa3ea600700 1 -- 172.21.15.111:0/3749587584 wait complete. 2024-04-24T22:31:33.445 INFO:teuthology.orchestra.run.smithi111.stdout:Fetching dashboard port number... 2024-04-24T22:31:35.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:34 smithi111 ceph-mon[27746]: mgrmap e13: x(active, since 6s) 2024-04-24T22:31:35.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:34 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2303976420' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T22:31:35.641 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T22:31:35.641 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.493+0000 7fabd3a00700 1 Processor -- start 2024-04-24T22:31:35.641 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.493+0000 7fabd3a00700 1 -- start start 2024-04-24T22:31:35.641 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.494+0000 7fabd3a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:35.641 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.494+0000 7fabd3a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc07f220 con 0x7fabcc07e830 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.494+0000 7fabd1000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.494+0000 7fabd1000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55158/0 (socket says 172.21.15.111:55158) 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.494+0000 7fabd1000700 1 -- 172.21.15.111:0/1403424875 learned_addr learned my addr 172.21.15.111:0/1403424875 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.495+0000 7fabd1000700 1 -- 172.21.15.111:0/1403424875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc07fa30 con 0x7fabcc07e830 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.495+0000 7fabd1000700 1 --2- 172.21.15.111:0/1403424875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fabc00098a0 tx=0x7fabc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b15e7e2c9fff71e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.495+0000 7fabcb400700 1 -- 172.21.15.111:0/1403424875 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc0004160 con 0x7fabcc07e830 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.495+0000 7fabcb400700 1 -- 172.21.15.111:0/1403424875 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc0004300 con 0x7fabcc07e830 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.495+0000 7fabcb400700 1 -- 172.21.15.111:0/1403424875 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc0011b20 con 0x7fabcc07e830 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 -- 172.21.15.111:0/1403424875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 msgr2=0x7fabcc07ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 --2- 172.21.15.111:0/1403424875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fabc00098a0 tx=0x7fabc0009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 -- 172.21.15.111:0/1403424875 shutdown_connections 2024-04-24T22:31:35.642 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 --2- 172.21.15.111:0/1403424875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc07ec50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 -- 172.21.15.111:0/1403424875 >> 172.21.15.111:0/1403424875 conn(0x7fabcc07d3a0 msgr2=0x7fabcc07d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 -- 172.21.15.111:0/1403424875 shutdown_connections 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.496+0000 7fabd3a00700 1 -- 172.21.15.111:0/1403424875 wait complete. 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.497+0000 7fabd3a00700 1 Processor -- start 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd3a00700 1 -- start start 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd3a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd3a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc080df0 con 0x7fabcc07e830 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd1000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd1000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55166/0 (socket says 172.21.15.111:55166) 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.498+0000 7fabd1000700 1 -- 172.21.15.111:0/2303976420 learned_addr learned my addr 172.21.15.111:0/2303976420 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabd1000700 1 -- 172.21.15.111:0/2303976420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc1a28c0 con 0x7fabcc07e830 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabd1000700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011b60 tx=0x7fabc0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc0003b60 con 0x7fabcc07e830 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc001a910 con 0x7fabcc07e830 2024-04-24T22:31:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc1a2b50 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc0024930 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.499+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc1a2fb0 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.500+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fabc0022020 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.501+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabcc063890 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.501+0000 7fabc9600700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 0x7fabb403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.501+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabc004d530 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.501+0000 7fabcbe00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 0x7fabb403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.502+0000 7fabcbe00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 0x7fabb403b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fabbc006fd0 tx=0x7fabbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.506+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabc000b3e0 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.731+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fabcc062e40 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.732+0000 7fabc9600700 1 -- 172.21.15.111:0/2303976420 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fabc0016260 con 0x7fabcc07e830 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 msgr2=0x7fabb403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 0x7fabb403b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fabbc006fd0 tx=0x7fabbc006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 msgr2=0x7fabcc1a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011b60 tx=0x7fabc0011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 shutdown_connections 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fabb4039140 0x7fabb403b600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 --2- 172.21.15.111:0/2303976420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabcc07e830 0x7fabcc1a2380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 >> 172.21.15.111:0/2303976420 conn(0x7fabcc07d3a0 msgr2=0x7fabcc198ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 shutdown_connections 2024-04-24T22:31:35.645 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:34.735+0000 7fabd3a00700 1 -- 172.21.15.111:0/2303976420 wait complete. 2024-04-24T22:31:35.666 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:31:35.666 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-04-24T22:31:35.689 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:31:35.689 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-04-24T22:31:35.689 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-04-24T22:31:35.689 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T22:31:35.696 INFO:teuthology.orchestra.run.smithi111.stdout:Ceph Dashboard is now available at: 2024-04-24T22:31:35.696 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:35.697 INFO:teuthology.orchestra.run.smithi111.stdout: URL: https://smithi111.front.sepia.ceph.com:8443/ 2024-04-24T22:31:35.697 INFO:teuthology.orchestra.run.smithi111.stdout: User: admin 2024-04-24T22:31:35.697 INFO:teuthology.orchestra.run.smithi111.stdout: Password: owvb1vcz4g 2024-04-24T22:31:35.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:35.698 INFO:teuthology.orchestra.run.smithi111.stdout:Saving cluster configuration to /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config directory 2024-04-24T22:31:35.700 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling autotune for osd_memory_target 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.295+0000 7fc4a1e00700 1 Processor -- start 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.295+0000 7fc4a1e00700 1 -- start start 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.295+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.295+0000 7fc4a1e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10aa00 con 0x7fc49c10a010 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.296+0000 7fc49aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.296+0000 7fc49aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56634/0 (socket says 172.21.15.111:56634) 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.296+0000 7fc49aa00700 1 -- 172.21.15.111:0/242725920 learned_addr learned my addr 172.21.15.111:0/242725920 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.296+0000 7fc49aa00700 1 -- 172.21.15.111:0/242725920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c10b1c0 con 0x7fc49c10a010 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.297+0000 7fc49aa00700 1 --2- 172.21.15.111:0/242725920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaeb655fdcef30db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.297+0000 7fc499600700 1 -- 172.21.15.111:0/242725920 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c004160 con 0x7fc49c10a010 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.297+0000 7fc499600700 1 -- 172.21.15.111:0/242725920 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48c004300 con 0x7fc49c10a010 2024-04-24T22:31:37.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.297+0000 7fc499600700 1 -- 172.21.15.111:0/242725920 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c011b20 con 0x7fc49c10a010 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 -- 172.21.15.111:0/242725920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 msgr2=0x7fc49c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/242725920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 -- 172.21.15.111:0/242725920 shutdown_connections 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/242725920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c10a430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 -- 172.21.15.111:0/242725920 >> 172.21.15.111:0/242725920 conn(0x7fc49c07d3a0 msgr2=0x7fc49c07d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 -- 172.21.15.111:0/242725920 shutdown_connections 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.298+0000 7fc4a1e00700 1 -- 172.21.15.111:0/242725920 wait complete. 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.299+0000 7fc4a1e00700 1 Processor -- start 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.299+0000 7fc4a1e00700 1 -- start start 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc4a1e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10e570 con 0x7fc49c10a010 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc49aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc49aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56640/0 (socket says 172.21.15.111:56640) 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc49aa00700 1 -- 172.21.15.111:0/2499277281 learned_addr learned my addr 172.21.15.111:0/2499277281 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:37.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.300+0000 7fc49aa00700 1 -- 172.21.15.111:0/2499277281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19e4f0 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc49aa00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c003b60 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48c01a910 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19e780 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c024930 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.301+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c19ebe0 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.302+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc48c022020 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.302+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c063890 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.303+0000 7fc4a0a00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 0x7fc48003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.303+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc48c04d490 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.303+0000 7fc49a000700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 0x7fc48003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.304+0000 7fc49a000700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 0x7fc48003b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.308+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc48c0212c0 con 0x7fc49c10a010 2024-04-24T22:31:37.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.530+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc49c1a15b0 con 0x7fc49c10a010 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.531+0000 7fc4a0a00700 1 -- 172.21.15.111:0/2499277281 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc48c01ab70 con 0x7fc49c10a010 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 msgr2=0x7fc48003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 0x7fc48003b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 msgr2=0x7fc49c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c011c10 comp rx=0 tx=0).stop 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 shutdown_connections 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc480039140 0x7fc48003b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 --2- 172.21.15.111:0/2499277281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc49c10a010 0x7fc49c19dfb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.534+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 >> 172.21.15.111:0/2499277281 conn(0x7fc49c07d3a0 msgr2=0x7fc49c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.535+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 shutdown_connections 2024-04-24T22:31:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:36.535+0000 7fc4a1e00700 1 -- 172.21.15.111:0/2499277281 wait complete. 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.872+0000 7f0855000700 1 Processor -- start 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.872+0000 7f0855000700 1 -- start start 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.872+0000 7f0855000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.872+0000 7f0855000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08501115d0 con 0x7f085010eca0 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.873+0000 7f084de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:38.953 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.873+0000 7f084de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56642/0 (socket says 172.21.15.111:56642) 2024-04-24T22:31:38.954 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.873+0000 7f084de00700 1 -- 172.21.15.111:0/553484816 learned_addr learned my addr 172.21.15.111:0/553484816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.874+0000 7f084de00700 1 -- 172.21.15.111:0/553484816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0850111750 con 0x7f085010eca0 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.874+0000 7f084de00700 1 --2- 172.21.15.111:0/553484816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f0840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271a24d8ae6a1b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f084ca00700 1 -- 172.21.15.111:0/553484816 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840004160 con 0x7f085010eca0 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f084ca00700 1 -- 172.21.15.111:0/553484816 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0840004300 con 0x7f085010eca0 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f084ca00700 1 -- 172.21.15.111:0/553484816 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840011b20 con 0x7f085010eca0 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f0855000700 1 -- 172.21.15.111:0/553484816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 msgr2=0x7f0850111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f0855000700 1 --2- 172.21.15.111:0/553484816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f0840009bb0 comp rx=0 tx=0).stop 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f0855000700 1 -- 172.21.15.111:0/553484816 shutdown_connections 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f0855000700 1 --2- 172.21.15.111:0/553484816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f0850111090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.875+0000 7f0855000700 1 -- 172.21.15.111:0/553484816 >> 172.21.15.111:0/553484816 conn(0x7f085007d3a0 msgr2=0x7f085007f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.876+0000 7f0855000700 1 -- 172.21.15.111:0/553484816 shutdown_connections 2024-04-24T22:31:38.959 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.876+0000 7f0855000700 1 -- 172.21.15.111:0/553484816 wait complete. 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.877+0000 7f0855000700 1 Processor -- start 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.877+0000 7f0855000700 1 -- start start 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.877+0000 7f0855000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.877+0000 7f0855000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0850112810 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.877+0000 7f084de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f084de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56658/0 (socket says 172.21.15.111:56658) 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f084de00700 1 -- 172.21.15.111:0/4226534221 learned_addr learned my addr 172.21.15.111:0/4226534221 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f084de00700 1 -- 172.21.15.111:0/4226534221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08501a28b0 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f084de00700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0840011b60 tx=0x7f0840011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840003b60 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084001a910 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.878+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08501a2b40 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.879+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840024930 con 0x7f085010eca0 2024-04-24T22:31:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.879+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501a2fa0 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.880+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0840022020 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.880+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0850063890 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.880+0000 7f0846a00700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 0x7f083403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.880+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f084004d470 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.881+0000 7f084d400700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 0x7f083403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.881+0000 7f084d400700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 0x7f083403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:37.886+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f084000b3e0 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.225+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f08501a5970 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.232+0000 7f0846a00700 1 -- 172.21.15.111:0/4226534221 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f0840016260 con 0x7f085010eca0 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 msgr2=0x7f083403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 0x7f083403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 msgr2=0x7f08501a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0840011b60 tx=0x7f0840011c40 comp rx=0 tx=0).stop 2024-04-24T22:31:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 shutdown_connections 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.234+0000 7f0855000700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0834039140 0x7f083403b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.235+0000 7f0855000700 1 --2- 172.21.15.111:0/4226534221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f085010eca0 0x7f08501a2370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.235+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 >> 172.21.15.111:0/4226534221 conn(0x7f085007d3a0 msgr2=0x7f085007df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.235+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 shutdown_connections 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-04-24T22:31:38.235+0000 7f0855000700 1 -- 172.21.15.111:0/4226534221 wait complete. 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T22:31:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: ceph telemetry on 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout:For more information see: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:38.963 INFO:teuthology.orchestra.run.smithi111.stdout:Bootstrap complete. 2024-04-24T22:31:39.055 INFO:tasks.cephadm:Fetching config... 2024-04-24T22:31:39.056 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:31:39.056 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T22:31:39.104 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T22:31:39.105 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:31:39.105 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T22:31:39.186 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T22:31:39.186 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:31:39.186 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T22:31:39.308 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T22:31:39.308 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:31:39.309 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T22:31:39.357 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T22:31:39.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T22:31:39.454 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:31:39.481 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:39 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4226534221' entity='client.admin' 2024-04-24T22:31:39.483 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T22:31:39.525 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:31:39.537 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T22:31:39.579 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxasqlgiGuguc1+SmwW6vfHtGvuZ96YoLuHNBIGSlx15u8D3+YPIFFBgnOFM17BWtuUIblpyOGV1MvDY3TPI2tzGul0JEaCZIU5lYfoAyADP76/Gb49gos7oPRSTHAg1qHpXWLPtwhPbGjbvvMNJE5wXyob4ejXN+TxTdCnxoMRjwRHurUIkvEct+Xx9zyJHtvar78gcc+hrL8GpdsnZffV1Jpj0amZ1+/YetuIQX3cnzhShm6wrPp3UZyxnBMGeN00eBNwexaAMA+KZ0ccqeBVpZ+yems2D84gy+wq1Ij1QtZZoythsuC8eEPiUfUqlBZpQVyj28Jt0woUbkQ02zFId6g2hOqlMGmA0mGp+VRP8m30FqILWKKK0uTsWDBcIDyiP1gfP/kRjl0KZQ9pb8Pd8+0kTT4Dul5PBgjUsBM/nrdysXVAuRJn2CEPncxStWK8whF4Oew7jzFbs3HBSUEH/rNCOjphJ48uTIYsG3vTkKdn9oBG/FPbD7Je8tIk2E= ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:31:39.592 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T22:31:40.001 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:31:42.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 -- 172.21.15.111:0/2610026773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 msgr2=0x7fb6e8074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:42.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 --2- 172.21.15.111:0/2610026773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8074660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009910 tx=0x7fb6d8009c20 comp rx=0 tx=0).stop 2024-04-24T22:31:42.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 -- 172.21.15.111:0/2610026773 shutdown_connections 2024-04-24T22:31:42.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 --2- 172.21.15.111:0/2610026773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8074660 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:42.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 -- 172.21.15.111:0/2610026773 >> 172.21.15.111:0/2610026773 conn(0x7fb6e80fd220 msgr2=0x7fb6e80ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:42.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 -- 172.21.15.111:0/2610026773 shutdown_connections 2024-04-24T22:31:42.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.705+0000 7fb6ee400700 1 -- 172.21.15.111:0/2610026773 wait complete. 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.706+0000 7fb6ee400700 1 Processor -- start 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6ee400700 1 -- start start 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6ee400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6ee400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8109f00 con 0x7fb6e8076200 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6e7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6e7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56672/0 (socket says 172.21.15.111:56672) 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.707+0000 7fb6e7400700 1 -- 172.21.15.111:0/3089518812 learned_addr learned my addr 172.21.15.111:0/3089518812 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:42.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6e7400700 1 -- 172.21.15.111:0/3089518812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e8199ac0 con 0x7fb6e8076200 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6e7400700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005950 tx=0x7fb6d8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d8026070 con 0x7fb6e8076200 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d800bc30 con 0x7fb6e8076200 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d8003b50 con 0x7fb6e8076200 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e8199d50 con 0x7fb6e8076200 2024-04-24T22:31:42.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.708+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e819ca40 con 0x7fb6e8076200 2024-04-24T22:31:42.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.709+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb6d8029070 con 0x7fb6e8076200 2024-04-24T22:31:42.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.709+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e810a680 con 0x7fb6e8076200 2024-04-24T22:31:42.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.710+0000 7fb6dfe00700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 0x7fb6cc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:42.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.710+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6d801e070 con 0x7fb6e8076200 2024-04-24T22:31:42.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.710+0000 7fb6e6a00700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 0x7fb6cc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:42.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.711+0000 7fb6e6a00700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 0x7fb6cc03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4006fd0 tx=0x7fb6d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:42.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.716+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb6d8013070 con 0x7fb6e8076200 2024-04-24T22:31:42.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.946+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb6e802dc40 con 0x7fb6e8076200 2024-04-24T22:31:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.952+0000 7fb6dfe00700 1 -- 172.21.15.111:0/3089518812 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb6d8030020 con 0x7fb6e8076200 2024-04-24T22:31:42.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.955+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 msgr2=0x7fb6cc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:42.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.955+0000 7fb6ee400700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 0x7fb6cc03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4006fd0 tx=0x7fb6d4006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:42.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.955+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 msgr2=0x7fb6e8199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:42.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.955+0000 7fb6ee400700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005950 tx=0x7fb6d8005030 comp rx=0 tx=0).stop 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 shutdown_connections 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6cc0390f0 0x7fb6cc03b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 --2- 172.21.15.111:0/3089518812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6e8076200 0x7fb6e8199580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 >> 172.21.15.111:0/3089518812 conn(0x7fb6e80fd220 msgr2=0x7fb6e80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 shutdown_connections 2024-04-24T22:31:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:42.956+0000 7fb6ee400700 1 -- 172.21.15.111:0/3089518812 wait complete. 2024-04-24T22:31:43.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3089518812' entity='client.admin' 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:43.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:43.921 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T22:31:43.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T22:31:44.330 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:31:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.825+0000 7fe2bde00700 1 -- 172.21.15.111:0/698015565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 msgr2=0x7fe2b8104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.825+0000 7fe2bde00700 1 --2- 172.21.15.111:0/698015565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8104280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009910 tx=0x7fe2a8009c20 comp rx=0 tx=0).stop 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.825+0000 7fe2bde00700 1 -- 172.21.15.111:0/698015565 shutdown_connections 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.825+0000 7fe2bde00700 1 --2- 172.21.15.111:0/698015565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8104280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.825+0000 7fe2bde00700 1 -- 172.21.15.111:0/698015565 >> 172.21.15.111:0/698015565 conn(0x7fe2b80ff450 msgr2=0x7fe2b8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.826+0000 7fe2bde00700 1 -- 172.21.15.111:0/698015565 shutdown_connections 2024-04-24T22:31:46.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.826+0000 7fe2bde00700 1 -- 172.21.15.111:0/698015565 wait complete. 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2bde00700 1 Processor -- start 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2bde00700 1 -- start start 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2bde00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2bde00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b8109eb0 con 0x7fe2b8103e60 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56040/0 (socket says 172.21.15.111:56040) 2024-04-24T22:31:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.827+0000 7fe2b6a00700 1 -- 172.21.15.111:0/141957207 learned_addr learned my addr 172.21.15.111:0/141957207 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2b6a00700 1 -- 172.21.15.111:0/141957207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b8199b50 con 0x7fe2b8103e60 2024-04-24T22:31:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2b6a00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005850 tx=0x7fe2a8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a8026070 con 0x7fe2b8103e60 2024-04-24T22:31:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a80044a0 con 0x7fe2b8103e60 2024-04-24T22:31:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a8003ab0 con 0x7fe2b8103e60 2024-04-24T22:31:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.828+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b8199de0 con 0x7fe2b8103e60 2024-04-24T22:31:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.829+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b819ca80 con 0x7fe2b8103e60 2024-04-24T22:31:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.830+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe2a8029070 con 0x7fe2b8103e60 2024-04-24T22:31:46.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.830+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b810a630 con 0x7fe2b8103e60 2024-04-24T22:31:46.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.830+0000 7fe2bca00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 0x7fe29c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:46.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.830+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe2a801e070 con 0x7fe2b8103e60 2024-04-24T22:31:46.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.830+0000 7fe2afe00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 0x7fe29c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:46.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.831+0000 7fe2afe00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 0x7fe29c03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0006fd0 tx=0x7fe2a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:46.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:46.836+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe2a8013070 con 0x7fe2b8103e60 2024-04-24T22:31:47.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.060+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe2b802dc40 con 0x7fe29c039150 2024-04-24T22:31:47.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.067+0000 7fe2bca00700 1 -- 172.21.15.111:0/141957207 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2b802dc40 con 0x7fe29c039150 2024-04-24T22:31:47.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 msgr2=0x7fe29c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:47.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 0x7fe29c03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0006fd0 tx=0x7fe2a0006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:47.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 msgr2=0x7fe2b8199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:47.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005850 tx=0x7fe2a8005030 comp rx=0 tx=0).stop 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 shutdown_connections 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe29c039150 0x7fe29c03b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 --2- 172.21.15.111:0/141957207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b8103e60 0x7fe2b8199610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 >> 172.21.15.111:0/141957207 conn(0x7fe2b80ff450 msgr2=0x7fe2b80fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 shutdown_connections 2024-04-24T22:31:47.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:47.070+0000 7fe2bde00700 1 -- 172.21.15.111:0/141957207 wait complete. 2024-04-24T22:31:47.866 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-04-24T22:31:47.866 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:31:47.867 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-04-24T22:31:47.885 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:31:47.885 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:31:47.945 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-04-24T22:31:47.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch host add smithi171 2024-04-24T22:31:48.356 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:48.500 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:48 smithi111 ceph-mon[27746]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:31:49.396 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:49 smithi111 ceph-mon[27746]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.539+0000 7f3191000700 1 -- 172.21.15.111:0/4193782597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 msgr2=0x7f318c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.539+0000 7f3191000700 1 --2- 172.21.15.111:0/4193782597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c100d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c009c20 comp rx=0 tx=0).stop 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.540+0000 7f3191000700 1 -- 172.21.15.111:0/4193782597 shutdown_connections 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.540+0000 7f3191000700 1 --2- 172.21.15.111:0/4193782597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c100d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.540+0000 7f3191000700 1 -- 172.21.15.111:0/4193782597 >> 172.21.15.111:0/4193782597 conn(0x7f318c0fc670 msgr2=0x7f318c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.540+0000 7f3191000700 1 -- 172.21.15.111:0/4193782597 shutdown_connections 2024-04-24T22:31:50.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.540+0000 7f3191000700 1 -- 172.21.15.111:0/4193782597 wait complete. 2024-04-24T22:31:50.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.541+0000 7f3191000700 1 Processor -- start 2024-04-24T22:31:50.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.541+0000 7f3191000700 1 -- start start 2024-04-24T22:31:50.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3191000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:50.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3191000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f318c10e4d0 con 0x7f318c100990 2024-04-24T22:31:50.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3189e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:50.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3189e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56054/0 (socket says 172.21.15.111:56054) 2024-04-24T22:31:50.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3189e00700 1 -- 172.21.15.111:0/713873030 learned_addr learned my addr 172.21.15.111:0/713873030 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:50.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.542+0000 7f3189e00700 1 -- 172.21.15.111:0/713873030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c19fb60 con 0x7f318c100990 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3189e00700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f317c005850 tx=0x7f317c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c026070 con 0x7f318c100990 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c19fdf0 con 0x7f318c100990 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317c00bc30 con 0x7f318c100990 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c003a20 con 0x7f318c100990 2024-04-24T22:31:50.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.543+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c1997f0 con 0x7f318c100990 2024-04-24T22:31:50.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.544+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f317c029070 con 0x7f318c100990 2024-04-24T22:31:50.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.544+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c06a930 con 0x7f318c100990 2024-04-24T22:31:50.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.545+0000 7f3182a00700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 0x7f317003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:50.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.545+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f317c01e070 con 0x7f318c100990 2024-04-24T22:31:50.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.545+0000 7f3189400700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 0x7f317003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:50.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.546+0000 7f3189400700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 0x7f317003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:50.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.551+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f317c013070 con 0x7f318c100990 2024-04-24T22:31:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:50.781+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f318c19acd0 con 0x7f31700390f0 2024-04-24T22:31:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:50 smithi111 ceph-mon[27746]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:31:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:50 smithi111 ceph-mon[27746]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:52.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:52 smithi111 ceph-mon[27746]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:31:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:52 smithi111 ceph-mon[27746]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:52 smithi111 ceph-mon[27746]: Deploying cephadm binary to smithi171 2024-04-24T22:31:54.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:54 smithi111 ceph-mon[27746]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:55.439 INFO:teuthology.orchestra.run.smithi111.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-04-24T22:31:55.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.437+0000 7f3182a00700 1 -- 172.21.15.111:0/713873030 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f318c19acd0 con 0x7f31700390f0 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 msgr2=0x7f317003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 0x7f317003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 msgr2=0x7f318c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f317c005850 tx=0x7f317c005030 comp rx=0 tx=0).stop 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 shutdown_connections 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31700390f0 0x7f317003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 --2- 172.21.15.111:0/713873030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c100990 0x7f318c19f620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 >> 172.21.15.111:0/713873030 conn(0x7f318c0fc670 msgr2=0x7f318c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:55.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.441+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 shutdown_connections 2024-04-24T22:31:55.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:55.442+0000 7f3191000700 1 -- 172.21.15.111:0/713873030 wait complete. 2024-04-24T22:31:56.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T22:31:56.780 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:31:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:56 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:56 smithi111 ceph-mon[27746]: Added host smithi171 2024-04-24T22:31:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:56 smithi111 ceph-mon[27746]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:56 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:31:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:56 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:57.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:58.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:58 smithi111 ceph-mon[27746]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:31:58.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:31:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:31:58.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.807+0000 7f211b800700 1 -- 172.21.15.111:0/2650756039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 msgr2=0x7f21141031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:58.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.807+0000 7f211b800700 1 --2- 172.21.15.111:0/2650756039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f21141031b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2108009910 tx=0x7f2108009c20 comp rx=0 tx=0).stop 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.808+0000 7f211b800700 1 -- 172.21.15.111:0/2650756039 shutdown_connections 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.808+0000 7f211b800700 1 --2- 172.21.15.111:0/2650756039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f21141031b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.808+0000 7f211b800700 1 -- 172.21.15.111:0/2650756039 >> 172.21.15.111:0/2650756039 conn(0x7f2114077800 msgr2=0x7f21140feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.808+0000 7f211b800700 1 -- 172.21.15.111:0/2650756039 shutdown_connections 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.808+0000 7f211b800700 1 -- 172.21.15.111:0/2650756039 wait complete. 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f211b800700 1 Processor -- start 2024-04-24T22:31:58.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f211b800700 1 -- start start 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f211b800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f211b800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f211410c2c0 con 0x7f2114102dd0 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f2118e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f2118e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51034/0 (socket says 172.21.15.111:51034) 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.809+0000 7f2118e00700 1 -- 172.21.15.111:0/3768943056 learned_addr learned my addr 172.21.15.111:0/3768943056 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:31:58.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f2118e00700 1 -- 172.21.15.111:0/3768943056 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211419fba0 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f2118e00700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2108006010 tx=0x7f2108005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2108026070 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210800bc30 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211419fe30 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.810+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21080039a0 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.811+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141997e0 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.812+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2114069f40 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.812+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2108029070 con 0x7f2114102dd0 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.812+0000 7f2111600700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 0x7f20fc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:31:58.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.812+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f210801e070 con 0x7f2114102dd0 2024-04-24T22:31:58.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.813+0000 7f2113e00700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 0x7f20fc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:31:58.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.813+0000 7f2113e00700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 0x7f20fc03b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2100006fd0 tx=0x7f2100006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:31:58.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:58.818+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2108013070 con 0x7f2114102dd0 2024-04-24T22:31:59.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.049+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2114063890 con 0x7f20fc039150 2024-04-24T22:31:59.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.051+0000 7f2111600700 1 -- 172.21.15.111:0/3768943056 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f2114063890 con 0x7f20fc039150 2024-04-24T22:31:59.054 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:31:59.054 INFO:teuthology.orchestra.run.smithi111.stdout:[{"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.054+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 msgr2=0x7f20fc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 0x7f20fc03b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2100006fd0 tx=0x7f2100006e40 comp rx=0 tx=0).stop 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 msgr2=0x7f211419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2108006010 tx=0x7f2108005030 comp rx=0 tx=0).stop 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 shutdown_connections 2024-04-24T22:31:59.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20fc039150 0x7f20fc03b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:59.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 --2- 172.21.15.111:0/3768943056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2114102dd0 0x7f211419f660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:31:59.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 >> 172.21.15.111:0/3768943056 conn(0x7f2114077800 msgr2=0x7f2114107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:31:59.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 shutdown_connections 2024-04-24T22:31:59.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:31:59.055+0000 7f211b800700 1 -- 172.21.15.111:0/3768943056 wait complete. 2024-04-24T22:31:59.821 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-04-24T22:31:59.822 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:31:59.822 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-04-24T22:31:59.840 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:31:59.840 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:31:59.898 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-04-24T22:31:59.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch host add smithi186 2024-04-24T22:32:00.339 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:00.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:00 smithi111 ceph-mon[27746]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:32:00.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:00 smithi111 ceph-mon[27746]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.360+0000 7fb635800700 1 -- 172.21.15.111:0/1149024439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 msgr2=0x7fb630102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.360+0000 7fb635800700 1 --2- 172.21.15.111:0/1149024439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb630102250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb620009910 tx=0x7fb620009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.360+0000 7fb635800700 1 -- 172.21.15.111:0/1149024439 shutdown_connections 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.360+0000 7fb635800700 1 --2- 172.21.15.111:0/1149024439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb630102250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.361+0000 7fb635800700 1 -- 172.21.15.111:0/1149024439 >> 172.21.15.111:0/1149024439 conn(0x7fb6300fd720 msgr2=0x7fb6300ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:02.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.361+0000 7fb635800700 1 -- 172.21.15.111:0/1149024439 shutdown_connections 2024-04-24T22:32:02.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.361+0000 7fb635800700 1 -- 172.21.15.111:0/1149024439 wait complete. 2024-04-24T22:32:02.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.362+0000 7fb635800700 1 Processor -- start 2024-04-24T22:32:02.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.362+0000 7fb635800700 1 -- start start 2024-04-24T22:32:02.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb635800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:02.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb635800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010c240 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb62ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb62ea00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51062/0 (socket says 172.21.15.111:51062) 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb62ea00700 1 -- 172.21.15.111:0/1422784578 learned_addr learned my addr 172.21.15.111:0/1422784578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.363+0000 7fb62ea00700 1 -- 172.21.15.111:0/1422784578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63019d990 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb62ea00700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb620006010 tx=0x7fb620005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb620026070 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62000bbd0 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63019dc80 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb620003910 con 0x7fb630101e70 2024-04-24T22:32:02.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.364+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb630197610 con 0x7fb630101e70 2024-04-24T22:32:02.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.365+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb630103350 con 0x7fb630101e70 2024-04-24T22:32:02.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.365+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb620029070 con 0x7fb630101e70 2024-04-24T22:32:02.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.366+0000 7fb627400700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 0x7fb61403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:02.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.366+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb62001e070 con 0x7fb630101e70 2024-04-24T22:32:02.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.366+0000 7fb627e00700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 0x7fb61403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:02.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.367+0000 7fb627e00700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 0x7fb61403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb618006fd0 tx=0x7fb618006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.371+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb620013070 con 0x7fb630101e70 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:02 smithi111 ceph-mon[27746]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:02.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:02.602+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) -- 0x7fb630063890 con 0x7fb614039150 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:03 smithi111 ceph-mon[27746]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:04 smithi111 ceph-mon[27746]: Deploying cephadm binary to smithi186 2024-04-24T22:32:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:04 smithi111 ceph-mon[27746]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:07.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:06 smithi111 ceph-mon[27746]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:07.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.283+0000 7fb627400700 1 -- 172.21.15.111:0/1422784578 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb630063890 con 0x7fb614039150 2024-04-24T22:32:07.286 INFO:teuthology.orchestra.run.smithi111.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-04-24T22:32:07.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 msgr2=0x7fb61403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:07.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 0x7fb61403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb618006fd0 tx=0x7fb618006e40 comp rx=0 tx=0).stop 2024-04-24T22:32:07.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 msgr2=0x7fb63019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:07.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb620006010 tx=0x7fb620005030 comp rx=0 tx=0).stop 2024-04-24T22:32:07.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 shutdown_connections 2024-04-24T22:32:07.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb614039150 0x7fb61403b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 --2- 172.21.15.111:0/1422784578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb630101e70 0x7fb63019d450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.287+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 >> 172.21.15.111:0/1422784578 conn(0x7fb6300fd720 msgr2=0x7fb630104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.288+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 shutdown_connections 2024-04-24T22:32:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:07.288+0000 7fb635800700 1 -- 172.21.15.111:0/1422784578 wait complete. 2024-04-24T22:32:08.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T22:32:08.531 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:08.685 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:08.685 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:08 smithi111 ceph-mon[27746]: Added host smithi186 2024-04-24T22:32:08.685 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:08.685 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:08 smithi111 ceph-mon[27746]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:08.685 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:09.740 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:10.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.568+0000 7f57d8200700 1 -- 172.21.15.111:0/1929054157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 msgr2=0x7f57d0104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:10.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.568+0000 7f57d8200700 1 --2- 172.21.15.111:0/1929054157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d0104460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b0a0 tx=0x7f57c400b3b0 comp rx=0 tx=0).stop 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.569+0000 7f57d8200700 1 -- 172.21.15.111:0/1929054157 shutdown_connections 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.569+0000 7f57d8200700 1 --2- 172.21.15.111:0/1929054157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d0104460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.569+0000 7f57d8200700 1 -- 172.21.15.111:0/1929054157 >> 172.21.15.111:0/1929054157 conn(0x7f57d00ff950 msgr2=0x7f57d0101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.569+0000 7f57d8200700 1 -- 172.21.15.111:0/1929054157 shutdown_connections 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.569+0000 7f57d8200700 1 -- 172.21.15.111:0/1929054157 wait complete. 2024-04-24T22:32:10.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.570+0000 7f57d8200700 1 Processor -- start 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.570+0000 7f57d8200700 1 -- start start 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d8200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d8200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010e450 con 0x7f57d0104080 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d5800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d5800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51116/0 (socket says 172.21.15.111:51116) 2024-04-24T22:32:10.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d5800700 1 -- 172.21.15.111:0/392799129 learned_addr learned my addr 172.21.15.111:0/392799129 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.571+0000 7f57d5800700 1 -- 172.21.15.111:0/392799129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d019fbe0 con 0x7f57d0104080 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57d5800700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57c40093e0 tx=0x7f57c4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c4017040 con 0x7f57d0104080 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c4022070 con 0x7f57d0104080 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c401d440 con 0x7f57d0104080 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d019fe70 con 0x7f57d0104080 2024-04-24T22:32:10.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.572+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d0199770 con 0x7f57d0104080 2024-04-24T22:32:10.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.573+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d0105560 con 0x7f57d0104080 2024-04-24T22:32:10.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.573+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f57c4004000 con 0x7f57d0104080 2024-04-24T22:32:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.574+0000 7f57ce000700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 0x7f57b803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.574+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f57c400f050 con 0x7f57d0104080 2024-04-24T22:32:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.574+0000 7f57d4e00700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 0x7f57b803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.575+0000 7f57d4e00700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 0x7f57b803b660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:10.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.579+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57c4007e20 con 0x7f57d0104080 2024-04-24T22:32:10.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:10 smithi111 ceph-mon[27746]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:10.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:10 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.813+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57d0063890 con 0x7f57b80391a0 2024-04-24T22:32:10.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.815+0000 7f57ce000700 1 -- 172.21.15.111:0/392799129 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f57d0063890 con 0x7f57b80391a0 2024-04-24T22:32:10.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:32:10.817 INFO:teuthology.orchestra.run.smithi111.stdout:[{"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-04-24T22:32:10.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 msgr2=0x7f57b803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:10.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 0x7f57b803b660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc006e40 comp rx=0 tx=0).stop 2024-04-24T22:32:10.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 msgr2=0x7f57d019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:10.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57c40093e0 tx=0x7f57c4003ce0 comp rx=0 tx=0).stop 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 shutdown_connections 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f57b80391a0 0x7f57b803b660 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 --2- 172.21.15.111:0/392799129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57d0104080 0x7f57d019f6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 >> 172.21.15.111:0/392799129 conn(0x7f57d00ff950 msgr2=0x7f57d0105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.818+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 shutdown_connections 2024-04-24T22:32:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:10.819+0000 7f57d8200700 1 -- 172.21.15.111:0/392799129 wait complete. 2024-04-24T22:32:11.596 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T22:32:11.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T22:32:12.029 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:12.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:12 smithi111 ceph-mon[27746]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:32:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:12 smithi111 ceph-mon[27746]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.161+0000 7ffae6c00700 1 -- 172.21.15.111:0/854572377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 msgr2=0x7ffae00749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 --2- 172.21.15.111:0/854572377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae00749b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009910 tx=0x7ffad0009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 -- 172.21.15.111:0/854572377 shutdown_connections 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 --2- 172.21.15.111:0/854572377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae00749b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 -- 172.21.15.111:0/854572377 >> 172.21.15.111:0/854572377 conn(0x7ffae00fd720 msgr2=0x7ffae00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 -- 172.21.15.111:0/854572377 shutdown_connections 2024-04-24T22:32:14.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.162+0000 7ffae6c00700 1 -- 172.21.15.111:0/854572377 wait complete. 2024-04-24T22:32:14.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.163+0000 7ffae6c00700 1 Processor -- start 2024-04-24T22:32:14.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.163+0000 7ffae6c00700 1 -- start start 2024-04-24T22:32:14.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.164+0000 7ffae6c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:14.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.164+0000 7ffae6c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffae01128b0 con 0x7ffae00745d0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.164+0000 7ffadfe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.164+0000 7ffadfe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51142/0 (socket says 172.21.15.111:51142) 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.164+0000 7ffadfe00700 1 -- 172.21.15.111:0/1564288228 learned_addr learned my addr 172.21.15.111:0/1564288228 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffadfe00700 1 -- 172.21.15.111:0/1564288228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae019e330 con 0x7ffae00745d0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffadfe00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffad0006010 tx=0x7ffad0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad0026070 con 0x7ffae00745d0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad000bc30 con 0x7ffae00745d0 2024-04-24T22:32:14.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae019e5c0 con 0x7ffae00745d0 2024-04-24T22:32:14.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad00039a0 con 0x7ffae00745d0 2024-04-24T22:32:14.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.165+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae019ea20 con 0x7ffae00745d0 2024-04-24T22:32:14.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.167+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ffad0029070 con 0x7ffae00745d0 2024-04-24T22:32:14.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.167+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae0075d10 con 0x7ffae00745d0 2024-04-24T22:32:14.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.167+0000 7ffadcc00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 0x7ffac403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:14.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.167+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffad001e070 con 0x7ffae00745d0 2024-04-24T22:32:14.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.167+0000 7ffadf400700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 0x7ffac403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:14.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.168+0000 7ffadf400700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 0x7ffac403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006fd0 tx=0x7ffacc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:14.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.173+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffad0013070 con 0x7ffae00745d0 2024-04-24T22:32:14.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-24T22:32:14.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:14.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:14 smithi111 ceph-mon[27746]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:14.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:14.411+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ffae019f150 con 0x7ffae00745d0 2024-04-24T22:32:15.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.345+0000 7ffadcc00700 1 -- 172.21.15.111:0/1564288228 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ffad0030090 con 0x7ffae00745d0 2024-04-24T22:32:15.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 msgr2=0x7ffac403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 0x7ffac403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006fd0 tx=0x7ffacc006e40 comp rx=0 tx=0).stop 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 msgr2=0x7ffae019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffad0006010 tx=0x7ffad0005030 comp rx=0 tx=0).stop 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 shutdown_connections 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffac4039150 0x7ffac403b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 --2- 172.21.15.111:0/1564288228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae00745d0 0x7ffae019ddf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 >> 172.21.15.111:0/1564288228 conn(0x7ffae00fd720 msgr2=0x7ffae00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 shutdown_connections 2024-04-24T22:32:15.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:15.350+0000 7ffae6c00700 1 -- 172.21.15.111:0/1564288228 wait complete. 2024-04-24T22:32:15.353 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:15.724 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:15 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1564288228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T22:32:16.139 INFO:tasks.cephadm:Adding mon.a on smithi111 2024-04-24T22:32:16.139 INFO:tasks.cephadm:Adding mon.b on smithi171 2024-04-24T22:32:16.139 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-04-24T22:32:16.140 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c' 2024-04-24T22:32:16.377 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:16.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:16 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1564288228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T22:32:16.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:16 smithi111 ceph-mon[27746]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:16.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:16 smithi111 ceph-mon[27746]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:18.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 -- 172.21.15.186:0/2526785713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 msgr2=0x7f292c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 --2- 172.21.15.186:0/2526785713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c102250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f291c009910 tx=0x7f291c009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 -- 172.21.15.186:0/2526785713 shutdown_connections 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 --2- 172.21.15.186:0/2526785713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c102250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 -- 172.21.15.186:0/2526785713 >> 172.21.15.186:0/2526785713 conn(0x7f292c0fd740 msgr2=0x7f292c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 -- 172.21.15.186:0/2526785713 shutdown_connections 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.038+0000 7f2930bf9700 1 -- 172.21.15.186:0/2526785713 wait complete. 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.039+0000 7f2930bf9700 1 Processor -- start 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.039+0000 7f2930bf9700 1 -- start start 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.039+0000 7f2930bf9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:18.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.039+0000 7f2930bf9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f292c10c240 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.040+0000 7f292a59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.040+0000 7f292a59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.186:46250/0 (socket says 172.21.15.186:46250) 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.040+0000 7f292a59c700 1 -- 172.21.15.186:0/1025451820 learned_addr learned my addr 172.21.15.186:0/1025451820 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.040+0000 7f292a59c700 1 -- 172.21.15.186:0/1025451820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c197c80 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f292a59c700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f291c005850 tx=0x7f291c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c026070 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c197f70 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c19c010 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291c00b6f0 con 0x7f292c101e70 2024-04-24T22:32:18.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.041+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c030060 con 0x7f292c101e70 2024-04-24T22:32:18.044 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.042+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c005330 con 0x7f292c101e70 2024-04-24T22:32:18.044 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.043+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f291c029070 con 0x7f292c101e70 2024-04-24T22:32:18.044 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.043+0000 7f291b7fe700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 0x7f291403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:18.044 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.043+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f291c01e070 con 0x7f292c101e70 2024-04-24T22:32:18.045 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.043+0000 7f2929d9b700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 0x7f291403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:18.045 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.044+0000 7f2929d9b700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 0x7f291403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:18.060 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.058+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f291c013070 con 0x7f292c101e70 2024-04-24T22:32:18.267 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.266+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) -- 0x7f290c000c80 con 0x7f29140390f0 2024-04-24T22:32:18.282 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.281+0000 7f291b7fe700 1 -- 172.21.15.186:0/1025451820 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f290c000c80 con 0x7f29140390f0 2024-04-24T22:32:18.283 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-04-24T22:32:18.284 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 msgr2=0x7f291403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:18.285 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 0x7f291403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920006e40 comp rx=0 tx=0).stop 2024-04-24T22:32:18.285 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 msgr2=0x7f292c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:18.285 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f291c005850 tx=0x7f291c005030 comp rx=0 tx=0).stop 2024-04-24T22:32:18.285 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 shutdown_connections 2024-04-24T22:32:18.285 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29140390f0 0x7f291403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:18.286 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 --2- 172.21.15.186:0/1025451820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f292c101e70 0x7f292c197740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:18.286 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 >> 172.21.15.186:0/1025451820 conn(0x7f292c0fd740 msgr2=0x7f292c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:18.286 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 shutdown_connections 2024-04-24T22:32:18.286 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:18.283+0000 7f2930bf9700 1 -- 172.21.15.186:0/1025451820 wait complete. 2024-04-24T22:32:18.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:18.917 DEBUG:teuthology.orchestra.run.smithi171:mon.b> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T22:32:18.920 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T22:32:18.922 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T22:32:18.922 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:32:19.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:19 smithi111 ceph-mon[27746]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:19.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:19 smithi111 ceph-mon[27746]: Saving service mon spec with placement smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c;count:3 2024-04-24T22:32:19.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:19 smithi111 ceph-mon[27746]: Deploying daemon mon.c on smithi186 2024-04-24T22:32:20.167 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:20.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:20 smithi111 ceph-mon[27746]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:22.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 -- 172.21.15.186:0/1032099713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 msgr2=0x7f7810106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 --2- 172.21.15.186:0/1032099713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810106fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7800009910 tx=0x7f7800009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 -- 172.21.15.186:0/1032099713 shutdown_connections 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 --2- 172.21.15.186:0/1032099713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810106fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 -- 172.21.15.186:0/1032099713 >> 172.21.15.186:0/1032099713 conn(0x7f7810077800 msgr2=0x7f78100feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 -- 172.21.15.186:0/1032099713 shutdown_connections 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.630+0000 7f781797d700 1 -- 172.21.15.186:0/1032099713 wait complete. 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.631+0000 7f781797d700 1 Processor -- start 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.631+0000 7f781797d700 1 -- start start 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.631+0000 7f781797d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.631+0000 7f781797d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f781010b1a0 con 0x7f7810106be0 2024-04-24T22:32:22.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.632+0000 7f7815719700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:22.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.632+0000 7f7815719700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.186:56830/0 (socket says 172.21.15.186:56830) 2024-04-24T22:32:22.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.632+0000 7f7815719700 1 -- 172.21.15.186:0/794915536 learned_addr learned my addr 172.21.15.186:0/794915536 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:32:22.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f7815719700 1 -- 172.21.15.186:0/794915536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7810197c60 con 0x7f7810106be0 2024-04-24T22:32:22.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f7815719700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7800005850 tx=0x7f7800005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:22.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7800026070 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7810197ef0 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781019bff0 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780000bc30 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.633+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78000039b0 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.634+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78101021e0 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.635+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7800029070 con 0x7f7810106be0 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.635+0000 7f78067fc700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 0x7f77fc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:22.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.635+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f780001e070 con 0x7f7810106be0 2024-04-24T22:32:22.638 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.635+0000 7f7814f18700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 0x7f77fc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:22.638 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.636+0000 7f7814f18700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 0x7f77fc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f780c006fd0 tx=0x7f780c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:22.651 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.650+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7800013070 con 0x7f7810106be0 2024-04-24T22:32:22.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:22 smithi111 ceph-mon[27746]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:22.937 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.935+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7810198a80 con 0x7f7810106be0 2024-04-24T22:32:22.938 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.936+0000 7f78067fc700 1 -- 172.21.15.186:0/794915536 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f780000bde0 con 0x7f7810106be0 2024-04-24T22:32:22.938 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:32:22.938 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:30:15.899552Z","created":"2024-04-24T22:30:15.899552Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.938+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 msgr2=0x7f77fc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.938+0000 7f781797d700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 0x7f77fc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f780c006fd0 tx=0x7f780c006e40 comp rx=0 tx=0).stop 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.938+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 msgr2=0x7f7810197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.938+0000 7f781797d700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7800005850 tx=0x7f7800005030 comp rx=0 tx=0).stop 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 shutdown_connections 2024-04-24T22:32:22.940 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77fc0390f0 0x7f77fc03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:22.941 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 --2- 172.21.15.186:0/794915536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7810106be0 0x7f7810197720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:22.941 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 >> 172.21.15.186:0/794915536 conn(0x7f7810077800 msgr2=0x7f7810069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:22.941 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 shutdown_connections 2024-04-24T22:32:22.941 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:22.939+0000 7f781797d700 1 -- 172.21.15.186:0/794915536 wait complete. 2024-04-24T22:32:22.941 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-24T22:32:23.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:23 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/794915536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:32:24.633 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T22:32:24.634 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:32:24.876 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:32:28.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:29.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: mon.a calling monitor election 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: mon.c calling monitor election 2024-04-24T22:32:29.039 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: monmap epoch 2 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: last_changed 2024-04-24T22:32:23.570172+0000 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: min_mon_release 19 (squid) 2024-04-24T22:32:29.040 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: election_strategy: 1 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: fsmap 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: mgrmap e13: x(active, since 61s) 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: overall HEALTH_OK 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:29.041 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 -- 172.21.15.186:0/4212984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c0035c0 msgr2=0x7fce2c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 --2- 172.21.15.186:0/4212984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c0035c0 0x7fce2c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce44079470 tx=0x7fce3400feb0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 -- 172.21.15.186:0/4212984221 shutdown_connections 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 --2- 172.21.15.186:0/4212984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c0035c0 0x7fce2c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 -- 172.21.15.186:0/4212984221 >> 172.21.15.186:0/4212984221 conn(0x7fce440772e0 msgr2=0x7fce440776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:29.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 -- 172.21.15.186:0/4212984221 shutdown_connections 2024-04-24T22:32:29.212 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.208+0000 7fce4a358700 1 -- 172.21.15.186:0/4212984221 wait complete. 2024-04-24T22:32:29.212 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 Processor -- start 2024-04-24T22:32:29.212 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 -- start start 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 0x7fce44101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce44109de0 con 0x7fce44101b40 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce4a358700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce44109c60 con 0x7fce2c0035c0 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce43fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce43fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47262/0 (socket says 172.21.15.186:47262) 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce43fff700 1 -- 172.21.15.186:0/1383048637 learned_addr learned my addr 172.21.15.186:0/1383048637 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:32:29.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.209+0000 7fce43fff700 1 -- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 msgr2=0x7fce44101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce43fff700 1 -- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 msgr2=0x7fce44101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce43fff700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce43fff700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce437fe700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 0x7fce44101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce437fe700 1 -- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 msgr2=0x7fce44101600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce437fe700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.210+0000 7fce437fe700 1 -- 172.21.15.186:0/1383048637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce44103520 con 0x7fce44101b40 2024-04-24T22:32:29.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce437fe700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 0x7fce44101fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b4d0 tx=0x7fce3800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fce38011630 con 0x7fce44101b40 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce441a3c20 con 0x7fce44101b40 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce38011cb0 con 0x7fce44101b40 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fce38010e60 con 0x7fce44101b40 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.211+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a4160 con 0x7fce44101b40 2024-04-24T22:32:29.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.212+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4406a190 con 0x7fce44101b40 2024-04-24T22:32:29.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.213+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fce38018020 con 0x7fce44101b40 2024-04-24T22:32:29.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.213+0000 7fce417fa700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 0x7fce3003b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:29.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.213+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce3804bb10 con 0x7fce44101b40 2024-04-24T22:32:29.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.213+0000 7fce43fff700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 0x7fce3003b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:29.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.214+0000 7fce43fff700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 0x7fce3003b300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce3400fa60 tx=0x7fce34005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:29.230 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.228+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce3801f020 con 0x7fce44101b40 2024-04-24T22:32:29.509 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.506+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fce441a4440 con 0x7fce44101b40 2024-04-24T22:32:29.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.507+0000 7fce417fa700 1 -- 172.21.15.186:0/1383048637 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fce3800f2e0 con 0x7fce44101b40 2024-04-24T22:32:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:32:29.510 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:32:23.570172Z","created":"2024-04-24T22:30:15.899552Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T22:32:29.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 msgr2=0x7fce3003b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 0x7fce3003b300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce3400fa60 tx=0x7fce34005a90 comp rx=0 tx=0).stop 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 msgr2=0x7fce44101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 0x7fce44101fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b4d0 tx=0x7fce3800b890 comp rx=0 tx=0).stop 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 shutdown_connections 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fce30038e40 0x7fce3003b300 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce2c0035c0 0x7fce44101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 --2- 172.21.15.186:0/1383048637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce44101b40 0x7fce44101fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.509+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 >> 172.21.15.186:0/1383048637 conn(0x7fce440772e0 msgr2=0x7fce440fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:29.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.510+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 shutdown_connections 2024-04-24T22:32:29.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:29.510+0000 7fce4a358700 1 -- 172.21.15.186:0/1383048637 wait complete. 2024-04-24T22:32:29.514 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-24T22:32:29.812 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:29 smithi186 ceph-mon[32753]: Deploying daemon mon.b on smithi171 2024-04-24T22:32:29.812 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:29 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/1383048637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:32:29.812 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:30.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:29 smithi111 ceph-mon[27746]: Deploying daemon mon.b on smithi171 2024-04-24T22:32:30.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:29 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/1383048637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:32:30.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:30 smithi186 ceph-mon[32753]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:30 smithi111 ceph-mon[27746]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:31.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:32:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:32:30.569+0000 7f34b7000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T22:32:31.114 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T22:32:31.115 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:32:31.364 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.db_log_dir: 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.wal_dir: 2024-04-24T22:32:32.713 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.write_buffer_manager: 0x55a43d9241e0 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:32:32.714 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:32:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:32 smithi186 ceph-mon[32753]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:32 smithi111 ceph-mon[27746]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.unordered_write: 0 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:32:33.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.row_cache: None 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.wal_filter: None 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.wal_compression: 0 2024-04-24T22:32:33.160 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:32:33.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:32:33.162 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:32:33.163 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_open_files: -1 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Compression algorithms supported: 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kZSTD supported: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:32:33.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.merge_operator: 2024-04-24T22:32:33.165 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_filter: None 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a43d951ba0) 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_type: 0 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: data_block_index_type: 0 2024-04-24T22:32:33.166 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_shortening: 1 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: checksum: 4 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: no_block_cache: 0 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache: 0x55a43d955090 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_options: 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: capacity : 536870912 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: num_shard_bits : 4 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: strict_capacity_limit : 0 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_compressed: (nil) 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: persistent_cache: (nil) 2024-04-24T22:32:33.167 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_size: 4096 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_size_deviation: 10 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_restart_interval: 16 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_block_restart_interval: 1 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: metadata_block_size: 4096 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: partition_filters: 0 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: use_delta_encoding: 1 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: filter_policy: bloomfilter 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: whole_key_filtering: 1 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: verify_compression: 0 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: format_version: 5 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: enable_index_compression: 1 2024-04-24T22:32:33.168 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_align: 0 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout: max_auto_readahead_size: 262144 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout: prepopulate_block_cache: 0 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout: initial_auto_readahead_size: 8192 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression: NoCompression 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.num_levels: 7 2024-04-24T22:32:33.169 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:32:33.170 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:32:33.171 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:32:33.172 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:32:33.173 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:32:33.174 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:32:33.175 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.ttl: 2592000 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T22:32:33.176 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T22:32:33.177 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6a54747c-7349-43b5-a4d7-834198313c5a 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997952712169, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997952712713, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713997952, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6a54747c-7349-43b5-a4d7-834198313c5a", "db_session_id": "IQ0OLDJ35QHEU0VQ5185", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713997952712812, "job": 1, "event": "recovery_finished"} 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a43da46000 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: DB pointer 0x55a43da30000 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T22:32:33.178 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** DB Stats ** 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T22:32:33.179 INFO:journalctl@ceph.mon.b.smithi171.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:32:33.180 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: Block cache BinnedLRUCache@0x55a43d955090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T22:32:33.181 INFO:journalctl@ceph.mon.b.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: starting mon.b rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(???) e0 preinit fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout: e1 2024-04-24T22:32:33.182 INFO:journalctl@ceph.mon.b.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout: legacy client fscid: -1 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout: No filesystems configured 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T22:32:33.183 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.184 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.185 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Deploying cephadm binary to smithi186 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Added host smithi186 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:33.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 bash[32319]: 2dee2d89540c280f3c2607a95651c6f962e44664a7f2a25d8b42e48bc49cf151 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.187 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.188 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1564288228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1564288228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:33.189 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Saving service mon spec with placement smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c;count:3 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Deploying daemon mon.c on smithi186 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/794915536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.a calling monitor election 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.190 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.c calling monitor election 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: monmap epoch 2 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: last_changed 2024-04-24T22:32:23.570172+0000 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: min_mon_release 19 (squid) 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: election_strategy: 1 2024-04-24T22:32:33.191 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: fsmap 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mgrmap e13: x(active, since 61s) 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: overall HEALTH_OK 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.192 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: Deploying daemon mon.b on smithi171 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/1383048637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:33.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 ceph-mon[32337]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T22:32:33.194 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:32 smithi171 systemd[1]: Started Ceph mon.b for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:32:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: mon.c calling monitor election 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: mon.a calling monitor election 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: monmap epoch 3 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: min_mon_release 19 (squid) 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: election_strategy: 1 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: fsmap 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: mgrmap e13: x(active, since 70s) 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: overall HEALTH_OK 2024-04-24T22:32:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.252 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.252 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:37 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: mon.c calling monitor election 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: mon.a calling monitor election 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: monmap epoch 3 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: min_mon_release 19 (squid) 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: election_strategy: 1 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: fsmap 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: mgrmap e13: x(active, since 70s) 2024-04-24T22:32:38.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: overall HEALTH_OK 2024-04-24T22:32:38.286 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.286 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.286 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:37 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:38.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 -- 172.21.15.186:0/99985860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6140062a0 msgr2=0x7ff614008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:38.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/99985860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6140062a0 0x7ff614008710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff628101490 tx=0x7ff618010ea0 comp rx=0 tx=0).stop 2024-04-24T22:32:38.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 -- 172.21.15.186:0/99985860 shutdown_connections 2024-04-24T22:32:38.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/99985860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6140062a0 0x7ff614008710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 -- 172.21.15.186:0/99985860 >> 172.21.15.186:0/99985860 conn(0x7ff6280772e0 msgr2=0x7ff6280776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 -- 172.21.15.186:0/99985860 shutdown_connections 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.855+0000 7ff62e0b4700 1 -- 172.21.15.186:0/99985860 wait complete. 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 Processor -- start 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 -- start start 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 0x7ff628197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff628197e80 0x7ff628198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff628107b90 con 0x7ff62819dcc0 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff628107a10 con 0x7ff628197e80 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff62e0b4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff628107d10 con 0x7ff6140062a0 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:44114/0 (socket says 172.21.15.186:44114) 2024-04-24T22:32:38.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 -- 172.21.15.186:0/439465711 learned_addr learned my addr 172.21.15.186:0/439465711 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff61ffff700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 0x7ff628197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 -- 172.21.15.186:0/439465711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff628197e80 msgr2=0x7ff628198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff628197e80 0x7ff628198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 -- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 msgr2=0x7ff628197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 0x7ff628197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff6277fe700 1 -- 172.21.15.186:0/439465711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a4650 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.856+0000 7ff61ffff700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 0x7ff628197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff6277fe700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff618005fd0 tx=0x7ff618005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6180050d0 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff618005270 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a4820 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618003a80 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.857+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a4d00 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.858+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff618003c20 con 0x7ff6140062a0 2024-04-24T22:32:38.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.858+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff628063890 con 0x7ff6140062a0 2024-04-24T22:32:38.862 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.858+0000 7ff6257fa700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 0x7ff60803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:38.862 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.858+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff61804f030 con 0x7ff6140062a0 2024-04-24T22:32:38.862 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.858+0000 7ff61ffff700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 0x7ff60803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:38.862 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.859+0000 7ff61ffff700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 0x7ff60803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6100079b0 tx=0x7ff610008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:38.864 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:38.863+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff61802a070 con 0x7ff6140062a0 2024-04-24T22:32:39.124 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:39.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:39.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:39.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.156+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff62802db30 con 0x7ff6140062a0 2024-04-24T22:32:39.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.156+0000 7ff6257fa700 1 -- 172.21.15.186:0/439465711 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7ff61800e3b0 con 0x7ff6140062a0 2024-04-24T22:32:39.158 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-24T22:32:39.158 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:32:32.736583Z","created":"2024-04-24T22:30:15.899552Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T22:32:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 msgr2=0x7ff60803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 0x7ff60803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6100079b0 tx=0x7ff610008040 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 msgr2=0x7ff62819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff618005fd0 tx=0x7ff618005750 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 shutdown_connections 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff608039580 0x7ff60803ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff628197e80 0x7ff628198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6140062a0 0x7ff62819d780 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 --2- 172.21.15.186:0/439465711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff62819dcc0 0x7ff628197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 >> 172.21.15.186:0/439465711 conn(0x7ff6280772e0 msgr2=0x7ff628104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 shutdown_connections 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:39.159+0000 7ff62e0b4700 1 -- 172.21.15.186:0/439465711 wait complete. 2024-04-24T22:32:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-04-24T22:32:39.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:39.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:39.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.090 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T22:32:40.090 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: mon.c calling monitor election 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: mon.a calling monitor election 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: monmap epoch 3 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: min_mon_release 19 (squid) 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: election_strategy: 1 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:40.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: fsmap 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: mgrmap e13: x(active, since 70s) 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: overall HEALTH_OK 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:40.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:40.494 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mon.b calling monitor election 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mon.b calling monitor election 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mon.c calling monitor election 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mon.a calling monitor election 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: monmap epoch 3 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: min_mon_release 19 (squid) 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: election_strategy: 1 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:40.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: fsmap 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: mgrmap e13: x(active, since 72s) 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: overall HEALTH_OK 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mon.b calling monitor election 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mon.b calling monitor election 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mon.c calling monitor election 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mon.a calling monitor election 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: monmap epoch 3 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: min_mon_release 19 (squid) 2024-04-24T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: election_strategy: 1 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: fsmap 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: mgrmap e13: x(active, since 72s) 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: overall HEALTH_OK 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.693 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mon.b calling monitor election 2024-04-24T22:32:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mon.b calling monitor election 2024-04-24T22:32:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mon.c calling monitor election 2024-04-24T22:32:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mon.a calling monitor election 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: monmap epoch 3 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: last_changed 2024-04-24T22:32:32.736583+0000 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: created 2024-04-24T22:30:15.899552+0000 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: min_mon_release 19 (squid) 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: election_strategy: 1 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: fsmap 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: mgrmap e13: x(active, since 72s) 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: overall HEALTH_OK 2024-04-24T22:32:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:40.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:41.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:41.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:32:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:42.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:42 smithi186 ceph-mon[32753]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T22:32:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:42 smithi186 ceph-mon[32753]: Reconfiguring daemon mon.a on smithi111 2024-04-24T22:32:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:42 smithi186 ceph-mon[32753]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:42 smithi111 ceph-mon[27746]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T22:32:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:42 smithi111 ceph-mon[27746]: Reconfiguring daemon mon.a on smithi111 2024-04-24T22:32:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:42 smithi111 ceph-mon[27746]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:42 smithi171 ceph-mon[32337]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T22:32:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:42 smithi171 ceph-mon[32337]: Reconfiguring daemon mon.a on smithi111 2024-04-24T22:32:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:42 smithi171 ceph-mon[32337]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 -- 172.21.15.111:0/479664628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de8076760 msgr2=0x7f7de8076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 --2- 172.21.15.111:0/479664628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de8076760 0x7f7de8076b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009910 tx=0x7f7ddc009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 -- 172.21.15.111:0/479664628 shutdown_connections 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 --2- 172.21.15.111:0/479664628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de8076760 0x7f7de8076b40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 -- 172.21.15.111:0/479664628 >> 172.21.15.111:0/479664628 conn(0x7f7de80ff950 msgr2=0x7f7de8101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:43.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 -- 172.21.15.111:0/479664628 shutdown_connections 2024-04-24T22:32:43.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.299+0000 7f7defa00700 1 -- 172.21.15.111:0/479664628 wait complete. 2024-04-24T22:32:43.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.300+0000 7f7defa00700 1 Processor -- start 2024-04-24T22:32:43.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 -- start start 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7de8076760 0x7f7de819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7de819e540 0x7f7de81a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81128b0 con 0x7f7de81a2e10 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8112730 con 0x7f7de8076760 2024-04-24T22:32:43.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7defa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8112a30 con 0x7f7de819e540 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7deda00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7deda00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49546/0 (socket says 172.21.15.111:49546) 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.301+0000 7f7deda00700 1 -- 172.21.15.111:0/2163534263 learned_addr learned my addr 172.21.15.111:0/2163534263 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7ded000700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7de8076760 0x7f7de819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7de7e00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7de819e540 0x7f7de81a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 -- 172.21.15.111:0/2163534263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7de819e540 msgr2=0x7f7de81a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7de819e540 0x7f7de81a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 -- 172.21.15.111:0/2163534263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7de8076760 msgr2=0x7f7de819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7de8076760 0x7f7de819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 -- 172.21.15.111:0/2163534263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de81aab80 con 0x7f7de81a2e10 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7deda00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba50 tx=0x7f7de000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.302+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0019070 con 0x7f7de81a2e10 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.303+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de000ce10 con 0x7f7de81a2e10 2024-04-24T22:32:43.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.303+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0007350 con 0x7f7de81a2e10 2024-04-24T22:32:43.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.303+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81aadb0 con 0x7f7de81a2e10 2024-04-24T22:32:43.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.303+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81ab210 con 0x7f7de81a2e10 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.304+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7de001c070 con 0x7f7de81a2e10 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.304+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de8063890 con 0x7f7de81a2e10 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.304+0000 7f7de5600700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 0x7f7dd003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.304+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7de004b670 con 0x7f7de81a2e10 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.305+0000 7f7ded000700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 0x7f7dd003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:43.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.305+0000 7f7ded000700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 0x7f7dd003ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:43.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.311+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7de00146f0 con 0x7f7de81a2e10 2024-04-24T22:32:43.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.540+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7de81ab670 con 0x7f7de81a2e10 2024-04-24T22:32:43.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.541+0000 7f7de5600700 1 -- 172.21.15.111:0/2163534263 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f7de000c930 con 0x7f7de81a2e10 2024-04-24T22:32:43.543 INFO:teuthology.orchestra.run.smithi111.stdout:# minimal ceph.conf for 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:43.543 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-04-24T22:32:43.543 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:32:43.543 INFO:teuthology.orchestra.run.smithi111.stdout: mon_host = [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-24T22:32:43.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 msgr2=0x7f7dd003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:43.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 0x7f7dd003ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc0058e0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 msgr2=0x7f7de81a32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba50 tx=0x7f7de000bd60 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 shutdown_connections 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7de81a2e10 0x7f7de81a32c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7dd0039580 0x7f7dd003ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7de8076760 0x7f7de819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 --2- 172.21.15.111:0/2163534263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7de819e540 0x7f7de81a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.544+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 >> 172.21.15.111:0/2163534263 conn(0x7f7de80ff950 msgr2=0x7f7de80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.545+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 shutdown_connections 2024-04-24T22:32:43.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:43.545+0000 7f7defa00700 1 -- 172.21.15.111:0/2163534263 wait complete. 2024-04-24T22:32:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2163534263' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2163534263' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:44.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2163534263' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:44.612 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T22:32:44.612 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:32:44.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T22:32:44.683 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:32:44.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:44.801 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:32:44.802 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T22:32:44.827 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:32:44.827 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:44.891 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:32:44.891 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T22:32:44.920 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:32:44.920 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:32:44.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:44.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:44.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:44.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:44.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:44.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:44.985 INFO:tasks.cephadm:Adding mgr.x on smithi111 2024-04-24T22:32:44.986 INFO:tasks.cephadm:Adding mgr.y on smithi171 2024-04-24T22:32:44.986 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-04-24T22:32:44.986 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi111=x;smithi171=y;smithi186=z' 2024-04-24T22:32:44.990 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:44.990 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:44.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:44.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:44.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:44.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:45.256 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:32:45.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:45 smithi186 ceph-mon[32753]: Reconfiguring mon.b (monmap changed)... 2024-04-24T22:32:45.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:45 smithi186 ceph-mon[32753]: Reconfiguring daemon mon.b on smithi171 2024-04-24T22:32:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:45 smithi111 ceph-mon[27746]: Reconfiguring mon.b (monmap changed)... 2024-04-24T22:32:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:45 smithi111 ceph-mon[27746]: Reconfiguring daemon mon.b on smithi171 2024-04-24T22:32:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:45 smithi171 ceph-mon[32337]: Reconfiguring mon.b (monmap changed)... 2024-04-24T22:32:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:45 smithi171 ceph-mon[32337]: Reconfiguring daemon mon.b on smithi171 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:47.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:32:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:32:47.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 -- 172.21.15.186:0/161152171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 msgr2=0x7fc074104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 --2- 172.21.15.186:0/161152171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074104c10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc064009910 tx=0x7fc064009c20 comp rx=0 tx=0).stop 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 -- 172.21.15.186:0/161152171 shutdown_connections 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 --2- 172.21.15.186:0/161152171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074104c10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 -- 172.21.15.186:0/161152171 >> 172.21.15.186:0/161152171 conn(0x7fc0740fc230 msgr2=0x7fc0740fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 -- 172.21.15.186:0/161152171 shutdown_connections 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.614+0000 7fc07949c700 1 -- 172.21.15.186:0/161152171 wait complete. 2024-04-24T22:32:47.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 Processor -- start 2024-04-24T22:32:47.617 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 -- start start 2024-04-24T22:32:47.619 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:47.619 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:47.619 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc074101190 0x7fc0741a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:47.619 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc074107d70 con 0x7fc074102820 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc074107bf0 con 0x7fc074101190 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc07949c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc074107ef0 con 0x7fc0741007d0 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:54388/0 (socket says 172.21.15.186:54388) 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 -- 172.21.15.186:0/3476714002 learned_addr learned my addr 172.21.15.186:0/3476714002 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc072ffd700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 -- 172.21.15.186:0/3476714002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc074101190 msgr2=0x7fc0741a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc074101190 0x7fc0741a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 -- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 msgr2=0x7fc074102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc0727fc700 1 -- 172.21.15.186:0/3476714002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0741a1b80 con 0x7fc0741007d0 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.615+0000 7fc072ffd700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc0727fc700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b780 tx=0x7fc06800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:47.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068004230 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0680043d0 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741a1e70 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068011c60 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.616+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a23f0 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.617+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc06801a400 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.617+0000 7fc05bfff700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 0x7fc05c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.617+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc06804c0c0 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.617+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074063890 con 0x7fc0741007d0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.617+0000 7fc072ffd700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 0x7fc05c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:47.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.618+0000 7fc072ffd700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 0x7fc05c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b5c0 tx=0x7fc06401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:47.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.622+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc068018510 con 0x7fc0741007d0 2024-04-24T22:32:47.838 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.837+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi171=y;smithi186=z", "target": ["mon-mgr", ""]}) -- 0x7fc074062730 con 0x7fc05c039580 2024-04-24T22:32:47.851 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.849+0000 7fc05bfff700 1 -- 172.21.15.186:0/3476714002 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc074062730 con 0x7fc05c039580 2024-04-24T22:32:47.851 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-04-24T22:32:47.853 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 msgr2=0x7fc05c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:47.853 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 0x7fc05c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b5c0 tx=0x7fc06401a040 comp rx=0 tx=0).stop 2024-04-24T22:32:47.853 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 msgr2=0x7fc074100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:47.853 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b780 tx=0x7fc06800bb40 comp rx=0 tx=0).stop 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 shutdown_connections 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc05c039580 0x7fc05c03ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc074101190 0x7fc0741a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc074102820 0x7fc074102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 --2- 172.21.15.186:0/3476714002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0741007d0 0x7fc074100c50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 >> 172.21.15.186:0/3476714002 conn(0x7fc0740fc230 msgr2=0x7fc0740fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 shutdown_connections 2024-04-24T22:32:47.854 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:32:47.852+0000 7fc07949c700 1 -- 172.21.15.186:0/3476714002 wait complete. 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: Reconfiguring mon.c (monmap changed)... 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: Reconfiguring daemon mon.c on smithi186 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: Reconfiguring mon.c (monmap changed)... 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: Reconfiguring daemon mon.c on smithi186 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: Reconfiguring mon.c (monmap changed)... 2024-04-24T22:32:48.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: Reconfiguring daemon mon.c on smithi186 2024-04-24T22:32:48.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.468 DEBUG:teuthology.orchestra.run.smithi171:mgr.y> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T22:32:48.471 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T22:32:48.473 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T22:32:48.473 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:32:48.473 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:32:48.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:32:48.526 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-04-24T22:32:48.574 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:32:48.574 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-24 22:31:44.328858291 +0000 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-24 22:25:43.670158597 +0000 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-24 22:25:43.670158597 +0000 2024-04-24T22:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-24 22:25:43.670158597 +0000 2024-04-24T22:32:48.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:32:48.646 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-24T22:32:48.647 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-24T22:32:48.647 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000390109 s, 1.3 MB/s 2024-04-24T22:32:48.649 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:32:48.704 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-04-24T22:32:48.757 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-24 22:31:44.387857266 +0000 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-24 22:25:44.238148659 +0000 2024-04-24T22:32:48.758 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-24 22:25:44.238148659 +0000 2024-04-24T22:32:48.759 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-24 22:25:44.238148659 +0000 2024-04-24T22:32:48.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:32:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-24T22:32:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-24T22:32:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000418646 s, 1.2 MB/s 2024-04-24T22:32:48.840 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:32:48.895 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-04-24T22:32:48.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:48.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi171=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: Saving service mgr spec with placement smithi111=x;smithi171=y;smithi186=z;count:3 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:48 smithi171 ceph-mon[32337]: Deploying daemon mgr.z on smithi186 2024-04-24T22:32:48.946 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:32:48.946 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:48.946 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T22:32:48.946 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:48.947 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:48.947 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-24 22:31:44.442856311 +0000 2024-04-24T22:32:48.947 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-24 22:25:44.844138057 +0000 2024-04-24T22:32:48.947 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-24 22:25:44.844138057 +0000 2024-04-24T22:32:48.947 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-24 22:25:44.844138057 +0000 2024-04-24T22:32:48.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:32:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi171=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: Saving service mgr spec with placement smithi111=x;smithi171=y;smithi186=z;count:3 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:48 smithi186 ceph-mon[32753]: Deploying daemon mgr.z on smithi186 2024-04-24T22:32:49.012 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:49.012 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi171=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:49.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: Saving service mgr spec with placement smithi111=x;smithi171=y;smithi186=z;count:3 2024-04-24T22:32:49.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:49.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:49.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:49.014 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:48 smithi111 ceph-mon[27746]: Deploying daemon mgr.z on smithi186 2024-04-24T22:32:49.020 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-24T22:32:49.020 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-24T22:32:49.020 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000349506 s, 1.5 MB/s 2024-04-24T22:32:49.024 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:32:49.076 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-24 22:31:44.503855251 +0000 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-24 22:25:45.450127454 +0000 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-24 22:25:45.450127454 +0000 2024-04-24T22:32:49.124 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-24 22:25:45.450127454 +0000 2024-04-24T22:32:49.125 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:32:49.248 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-24T22:32:49.248 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-24T22:32:49.248 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000379764 s, 1.3 MB/s 2024-04-24T22:32:49.252 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:32:49.310 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-24T22:32:49.310 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:32:49.329 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:32:49.330 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-24T22:32:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:32:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:49.387 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-24 22:32:32.161613117 +0000 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-24 22:24:47.541967367 +0000 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-24 22:24:47.541967367 +0000 2024-04-24T22:32:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-24 22:24:47.541967367 +0000 2024-04-24T22:32:49.388 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:32:49.453 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-24T22:32:49.550 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-24T22:32:49.551 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000284372 s, 1.8 MB/s 2024-04-24T22:32:49.552 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:32:49.617 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-24T22:32:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-24 22:32:32.161613117 +0000 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-24 22:24:47.876960901 +0000 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-24 22:24:47.876960901 +0000 2024-04-24T22:32:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-24 22:24:47.876960901 +0000 2024-04-24T22:32:49.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:32:49.752 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-24T22:32:49.752 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-24T22:32:49.752 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00019777 s, 2.6 MB/s 2024-04-24T22:32:49.753 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:32:49.814 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-24 22:32:32.161613117 +0000 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-24 22:24:48.226954145 +0000 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-24 22:24:48.226954145 +0000 2024-04-24T22:32:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-24 22:24:48.226954145 +0000 2024-04-24T22:32:49.873 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:32:49.914 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:49 smithi186 ceph-mon[32753]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:49.937 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-24T22:32:49.938 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-24T22:32:49.938 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000225735 s, 2.3 MB/s 2024-04-24T22:32:49.939 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:32:49.996 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-24T22:32:50.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:49 smithi111 ceph-mon[27746]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:50.052 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:32:50.052 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:50.052 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T22:32:50.052 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:50.053 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:50.053 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-24 22:32:32.162613098 +0000 2024-04-24T22:32:50.053 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-24 22:24:48.572947466 +0000 2024-04-24T22:32:50.053 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-24 22:24:48.572947466 +0000 2024-04-24T22:32:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-24 22:24:48.572947466 +0000 2024-04-24T22:32:50.053 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:32:50.113 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:49 smithi171 ceph-mon[32337]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:50.116 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-24T22:32:50.116 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-24T22:32:50.116 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000181015 s, 2.8 MB/s 2024-04-24T22:32:50.117 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:32:50.174 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-24T22:32:50.174 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:32:50.190 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:32:50.190 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-24T22:32:50.245 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:32:50.245 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-24 22:32:49.854587205 +0000 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-24 22:24:43.748560835 +0000 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-24 22:24:43.748560835 +0000 2024-04-24T22:32:50.246 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-24 22:24:43.748560835 +0000 2024-04-24T22:32:50.246 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:32:50.309 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-24T22:32:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-24T22:32:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000193789 s, 2.6 MB/s 2024-04-24T22:32:50.311 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:32:50.366 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-24 22:32:49.855587187 +0000 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-24 22:24:44.145553506 +0000 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-24 22:24:44.145553506 +0000 2024-04-24T22:32:50.422 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-24 22:24:44.145553506 +0000 2024-04-24T22:32:50.423 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:32:50.482 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 podman[34406]: 2024-04-24 22:32:50.461700224 +0000 UTC m=+0.446524493 container start 203973cd9bea991642c81d2b0838f360f0afcab80f1fdf725702002ff32f2634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:32:50.484 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-24T22:32:50.485 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-24T22:32:50.485 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000263719 s, 1.9 MB/s 2024-04-24T22:32:50.486 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:32:50.542 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-24 22:32:49.855587187 +0000 2024-04-24T22:32:50.599 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-24 22:24:44.518546620 +0000 2024-04-24T22:32:50.600 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-24 22:24:44.518546620 +0000 2024-04-24T22:32:50.600 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-24 22:24:44.518546620 +0000 2024-04-24T22:32:50.600 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:32:50.662 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-24T22:32:50.663 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-24T22:32:50.663 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000177241 s, 2.9 MB/s 2024-04-24T22:32:50.664 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:32:50.720 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-24T22:32:50.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 ceph-mgr[34510]: -- 172.21.15.186:0/2972979187 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562732bdc5a0 con 0x562732bb0c00 2024-04-24T22:32:50.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 bash[34406]: 203973cd9bea991642c81d2b0838f360f0afcab80f1fdf725702002ff32f2634 2024-04-24T22:32:50.750 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 systemd[1]: Started Ceph mgr.z for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:32:50.750 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:50.615+0000 7fc627d15200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-24 22:32:49.855587187 +0000 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-24 22:24:44.882539900 +0000 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-24 22:24:44.882539900 +0000 2024-04-24T22:32:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-24 22:24:44.882539900 +0000 2024-04-24T22:32:50.765 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:32:50.826 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-24T22:32:50.827 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-24T22:32:50.827 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000199254 s, 2.6 MB/s 2024-04-24T22:32:50.828 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:32:50.883 INFO:tasks.cephadm:Deploying osd.0 on smithi111 with /dev/vg_nvme/lv_4... 2024-04-24T22:32:50.884 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T22:32:51.227 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:50.813+0000 7fc627d15200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:32:51.227 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:50.919+0000 7fc627d15200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:32:51.227 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:51.225+0000 7fc627d15200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:32:51.308 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:51.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:51.309+0000 7fc627d15200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:51.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:51 smithi111 ceph-mon[27746]: Deploying daemon mgr.y on smithi171 2024-04-24T22:32:51.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:51 smithi171 ceph-mon[32337]: Deploying daemon mgr.y on smithi171 2024-04-24T22:32:52.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:52.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:52.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:52.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:52.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:32:52.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:52.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:52.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:51 smithi186 ceph-mon[32753]: Deploying daemon mgr.y on smithi171 2024-04-24T22:32:52.657 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 systemd[1]: Starting Ceph mgr.y for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:32:52.888 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:52 smithi186 ceph-mon[32753]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:52.888 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:52.534+0000 7fc627d15200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:32:52.888 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:52.709+0000 7fc627d15200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:32:52.888 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:52.886+0000 7fc627d15200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:32:52.978 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:52 smithi171 ceph-mon[32337]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:52.979 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 podman[33740]: 2024-04-24 22:32:52.640565068 +0000 UTC m=+0.017896364 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:32:52.979 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 podman[33740]: 2024-04-24 22:32:52.750752592 +0000 UTC m=+0.128083909 container create 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:32:52.979 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 podman[33740]: 2024-04-24 22:32:52.948131904 +0000 UTC m=+0.325463179 container init 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T22:32:52.979 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 podman[33740]: 2024-04-24 22:32:52.950800211 +0000 UTC m=+0.328131491 container start 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:32:52.979 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:52 smithi171 ceph-mgr[33759]: -- 172.21.15.171:0/2121536027 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d031efa5a0 con 0x55d031ece800 2024-04-24T22:32:52.980 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:52 smithi111 ceph-mon[27746]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:53.294 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 bash[33740]: 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 2024-04-24T22:32:53.294 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 systemd[1]: Started Ceph mgr.y for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:32:53.294 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:53.105+0000 7fa915539200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:32:53.294 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:53.291+0000 7fa915539200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:32:53.487 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:53.184+0000 7fc627d15200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:32:53.488 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:53.276+0000 7fc627d15200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:32:53.488 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:53.485+0000 7fc627d15200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:32:53.656 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:53.396+0000 7fa915539200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:32:53.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:53.688+0000 7fc627d15200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:32:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T22:32:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T22:32:54.040 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-24T22:32:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-24T22:32:54.041 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356001 s, 295 MB/s 2024-04-24T22:32:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-24T22:32:54.092 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:53.722+0000 7fa915539200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:32:54.092 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:53.811+0000 7fa915539200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:32:54.354 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.354 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.355 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.355 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.355 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:54.355 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-mon[32753]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:54 smithi171 ceph-mon[32337]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:54.435 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:54 smithi111 ceph-mon[27746]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:54.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:54.351+0000 7fc627d15200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:32:54.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-04-24T22:32:55.092 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:54.792+0000 7fc627d15200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:32:55.093 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:54.889+0000 7fc627d15200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:32:55.093 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:55.091+0000 7fc627d15200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:32:55.349 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:32:55.406 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:55.074+0000 7fa915539200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:32:55.406 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:55.244+0000 7fa915539200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:32:55.684 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:55.415+0000 7fa915539200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:32:55.834 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.834 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:55.835 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:55.613+0000 7fc627d15200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:55.705+0000 7fc627d15200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:55.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:56.004 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:55.703+0000 7fa915539200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:32:56.004 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:55.797+0000 7fa915539200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:32:56.004 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:56.002+0000 7fa915539200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:32:56.004 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:32:56.005 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:56.249 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:55.947+0000 7fc627d15200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:32:56.249 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:56.079+0000 7fc627d15200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:32:56.249 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:56.173+0000 7fc627d15200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:32:56.406 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:56.194+0000 7fa915539200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:32:56.685 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:56.288+0000 7fc627d15200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:32:56.686 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:56.380+0000 7fc627d15200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:32:56.828 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:56 smithi111 ceph-mon[27746]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T22:32:56.828 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:56 smithi111 ceph-mon[27746]: Reconfiguring daemon mgr.x on smithi111 2024-04-24T22:32:56.828 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:56 smithi111 ceph-mon[27746]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-mon[32753]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T22:32:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-mon[32753]: Reconfiguring daemon mgr.x on smithi111 2024-04-24T22:32:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:56 smithi186 ceph-mon[32753]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-mon[32337]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T22:32:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-mon[32337]: Reconfiguring daemon mgr.x on smithi111 2024-04-24T22:32:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-mon[32337]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:57.156 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:56 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:56.800+0000 7fa915539200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:32:57.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:57.194+0000 7fc627d15200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:32:57.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:32:57.409+0000 7fc627d15200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:32:57.517 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:57.218+0000 7fa915539200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:32:57.518 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:57.314+0000 7fa915539200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:32:57.518 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:57.516+0000 7fa915539200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: Standby manager daemon z started 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:32:57.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: Standby manager daemon z started 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:32:57.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: Standby manager daemon z started 2024-04-24T22:32:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:32:58.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:32:58.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:32:58.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/1455760958' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:32:58.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:58.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.125+0000 7f7d5de00700 1 -- 172.21.15.111:0/1559188896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58105640 msgr2=0x7f7d58107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.125+0000 7f7d5de00700 1 --2- 172.21.15.111:0/1559188896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58105640 0x7f7d58107a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009990 tx=0x7f7d44009ca0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 -- 172.21.15.111:0/1559188896 shutdown_connections 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 --2- 172.21.15.111:0/1559188896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58107f70 0x7f7d5810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 --2- 172.21.15.111:0/1559188896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58102d10 0x7f7d58105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 --2- 172.21.15.111:0/1559188896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58105640 0x7f7d58107a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 -- 172.21.15.111:0/1559188896 >> 172.21.15.111:0/1559188896 conn(0x7f7d580fc720 msgr2=0x7f7d580feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 -- 172.21.15.111:0/1559188896 shutdown_connections 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.126+0000 7f7d5de00700 1 -- 172.21.15.111:0/1559188896 wait complete. 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.127+0000 7f7d5de00700 1 Processor -- start 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.127+0000 7f7d5de00700 1 -- start start 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58102d10 0x7f7d58102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:58.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58107f70 0x7f7d58101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810dea0 con 0x7f7d58102d10 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810dd20 con 0x7f7d58105640 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d5de00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810e020 con 0x7f7d58107f70 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41774/0 (socket says 172.21.15.111:41774) 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 -- 172.21.15.111:0/3894117170 learned_addr learned my addr 172.21.15.111:0/3894117170 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56a00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58102d10 0x7f7d58102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d57400700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58107f70 0x7f7d58101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 -- 172.21.15.111:0/3894117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58107f70 msgr2=0x7f7d58101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58107f70 0x7f7d58101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 -- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58102d10 msgr2=0x7f7d58102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58102d10 0x7f7d58102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:32:58.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.128+0000 7f7d56000700 1 -- 172.21.15.111:0/3894117170 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d58101ab0 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d56000700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b7b0 tx=0x7f7d4400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44027070 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d44003d70 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d581a2640 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400f460 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.129+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d581a2a90 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.131+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d58068f20 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.131+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f7d4402a070 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.133+0000 7f7d5ca00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 0x7f7d3c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.133+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d4408d9e0 con 0x7f7d58105640 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.133+0000 7f7d56a00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 0x7f7d3c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.134+0000 7f7d56a00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 0x7f7d3c070420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009c60 tx=0x7f7d48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:32:58.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.137+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d44013070 con 0x7f7d58105640 2024-04-24T22:32:58.381 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.041+0000 7fa915539200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:32:58.381 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.133+0000 7fa915539200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:32:58.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:32:58.409+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7d5802ded0 con 0x7f7d3c06df60 2024-04-24T22:32:58.656 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.379+0000 7fa915539200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:32:58.656 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.520+0000 7fa915539200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:32:58.657 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.619+0000 7fa915539200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:32:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:32:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T22:32:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:32:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:59.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:59.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:32:59.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:32:59.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:59.156 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.741+0000 7fa915539200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:32:59.156 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:58.839+0000 7fa915539200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:32:59.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:32:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:59.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:59 smithi186 ceph-mon[32753]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:32:59.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:32:59.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:32:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:00.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:59 smithi111 ceph-mon[27746]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:00.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:00.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:00.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:32:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:59.678+0000 7fa915539200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:32:59.913+0000 7fa915539200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-mon[32337]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:32:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:00.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:00.717+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f7d4408fd10 con 0x7f7d58105640 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: Standby manager daemon y started 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:33:00.848 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:00 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:33:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: Standby manager daemon y started 2024-04-24T22:33:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:33:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:33:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:33:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:00 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:33:01.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:01.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: Standby manager daemon y started 2024-04-24T22:33:01.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:33:01.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:33:01.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:33:01.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:00 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/2067789991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:33:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:01 smithi186 ceph-mon[32753]: mgrmap e15: x(active, since 93s), standbys: y, z 2024-04-24T22:33:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:33:02.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:01 smithi111 ceph-mon[27746]: mgrmap e15: x(active, since 93s), standbys: y, z 2024-04-24T22:33:02.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:33:02.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:01 smithi171 ceph-mon[32337]: mgrmap e15: x(active, since 93s), standbys: y, z 2024-04-24T22:33:02.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:33:02.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:02.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1776468102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:02.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:02.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]': finished 2024-04-24T22:33:02.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T22:33:02.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:02 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1776468102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]': finished 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T22:33:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:02 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:03.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:03.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1776468102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:03.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]: dispatch 2024-04-24T22:33:03.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8"}]': finished 2024-04-24T22:33:03.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T22:33:03.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:02 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:03.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:03 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2314659405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:04.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:03 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2314659405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:04.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:03 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2314659405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:04.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:04 smithi186 ceph-mon[32753]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:05.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:04 smithi111 ceph-mon[27746]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:05.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:04 smithi171 ceph-mon[32337]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:06.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:05 smithi111 ceph-mon[27746]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:06.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:05 smithi171 ceph-mon[32337]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:06.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:05 smithi186 ceph-mon[32753]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:08.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:08 smithi111 ceph-mon[27746]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:08 smithi171 ceph-mon[32337]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:08 smithi186 ceph-mon[32753]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:10.770 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:10 smithi111 ceph-mon[27746]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:10 smithi171 ceph-mon[32337]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:10 smithi186 ceph-mon[32753]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:12.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:12 smithi111 ceph-mon[27746]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:12.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:33:12.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:12 smithi171 ceph-mon[32337]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:33:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:12 smithi186 ceph-mon[32753]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:33:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:13.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:13 smithi111 ceph-mon[27746]: Deploying daemon osd.0 on smithi111 2024-04-24T22:33:13.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:13 smithi186 ceph-mon[32753]: Deploying daemon osd.0 on smithi111 2024-04-24T22:33:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:13 smithi171 ceph-mon[32337]: Deploying daemon osd.0 on smithi111 2024-04-24T22:33:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:14 smithi186 ceph-mon[32753]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:14.812 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:14 smithi111 ceph-mon[27746]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:14.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:14 smithi171 ceph-mon[32337]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:16.768 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:16 smithi111 ceph-mon[27746]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:16 smithi171 ceph-mon[32337]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:16.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:16 smithi186 ceph-mon[32753]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:18.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:18 smithi171 ceph-mon[32337]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:18.989 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:18 smithi111 ceph-mon[27746]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:18 smithi186 ceph-mon[32753]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:20.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:20 smithi171 ceph-mon[32337]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:20.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:20.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:20.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:20 smithi186 ceph-mon[32753]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:20 smithi111 ceph-mon[27746]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:22.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:22 smithi111 ceph-mon[27746]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:22 smithi171 ceph-mon[32337]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:22 smithi186 ceph-mon[32753]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:24.687 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.688 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:24 smithi111 ceph-mon[27746]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:33:24.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.798+0000 7f7d5ca00700 1 -- 172.21.15.111:0/3894117170 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7d5802ded0 con 0x7f7d3c06df60 2024-04-24T22:33:24.802 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 0 on host 'smithi111' 2024-04-24T22:33:24.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 msgr2=0x7f7d3c070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 0x7f7d3c070420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009c60 tx=0x7f7d48009260 comp rx=0 tx=0).stop 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 msgr2=0x7f7d58100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b7b0 tx=0x7f7d4400bbb0 comp rx=0 tx=0).stop 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 shutdown_connections 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d58107f70 0x7f7d58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d58102d10 0x7f7d58102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7d3c06df60 0x7f7d3c070420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 --2- 172.21.15.111:0/3894117170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d58105640 0x7f7d58100cc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.802+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 >> 172.21.15.111:0/3894117170 conn(0x7f7d580fc720 msgr2=0x7f7d58103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:33:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.803+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 shutdown_connections 2024-04-24T22:33:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:33:24.803+0000 7f7d5de00700 1 -- 172.21.15.111:0/3894117170 wait complete. 2024-04-24T22:33:24.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:24 smithi171 ceph-mon[32337]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:24 smithi186 ceph-mon[32753]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:33:25.753 DEBUG:teuthology.orchestra.run.smithi111:osd.0> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T22:33:25.758 INFO:tasks.cephadm:Deploying osd.1 on smithi171 with /dev/vg_nvme/lv_4... 2024-04-24T22:33:25.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T22:33:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:33:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T22:33:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:33:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:25.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:33:25.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T22:33:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:33:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:26.005 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:33:26.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:26.640 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-mon[27746]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:26.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-mon[27746]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:33:26.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-mon[27746]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T22:33:26.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.641 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0[42944]: 2024-04-24T22:33:26.621+0000 7f2936400700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T22:33:26.641 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:33:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0[42944]: 2024-04-24T22:33:26.638+0000 7f292cc00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:33:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:26 smithi171 ceph-mon[32337]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:26 smithi171 ceph-mon[32337]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:33:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:26 smithi171 ceph-mon[32337]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T22:33:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:26 smithi186 ceph-mon[32753]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:26 smithi186 ceph-mon[32753]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:33:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:26 smithi186 ceph-mon[32753]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T22:33:27.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:27.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:27.822 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:27 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:33:27.822 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:27 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:33:27.822 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:27.822 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:27 smithi111 ceph-mon[27746]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' 2024-04-24T22:33:27.851 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:27 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:33:27.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:27 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:33:27.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:27.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:27 smithi171 ceph-mon[32337]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113034 s, 928 MB/s 2024-04-24T22:33:27.852 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-24T22:33:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:27 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:33:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:27 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:33:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:27 smithi186 ceph-mon[32753]: from='osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049]' entity='osd.0' 2024-04-24T22:33:28.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-04-24T22:33:28.707 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:33:28.750 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:28 smithi171 ceph-mon[32337]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:28.750 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:28.750 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:28 smithi171 ceph-mon[32337]: osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] boot 2024-04-24T22:33:28.750 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:28 smithi171 ceph-mon[32337]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T22:33:28.750 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:28 smithi186 ceph-mon[32753]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:28 smithi186 ceph-mon[32753]: osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] boot 2024-04-24T22:33:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:28 smithi186 ceph-mon[32753]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T22:33:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:28 smithi111 ceph-mon[27746]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:33:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:28 smithi111 ceph-mon[27746]: osd.0 [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] boot 2024-04-24T22:33:29.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:28 smithi111 ceph-mon[27746]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T22:33:29.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:33:29.690 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:29 smithi171 ceph-mon[32337]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T22:33:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:29 smithi186 ceph-mon[32753]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T22:33:30.018 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:29 smithi111 ceph-mon[27746]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T22:33:30.156 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 -- 172.21.15.171:0/2139101273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103490 msgr2=0x7f43101038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 --2- 172.21.15.171:0/2139101273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103490 0x7f43101038f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f430c0099c0 tx=0x7f430c009cd0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 -- 172.21.15.171:0/2139101273 shutdown_connections 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 --2- 172.21.15.171:0/2139101273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103e30 0x7f431010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 --2- 172.21.15.171:0/2139101273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f43101026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 --2- 172.21.15.171:0/2139101273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103490 0x7f43101038f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 -- 172.21.15.171:0/2139101273 >> 172.21.15.171:0/2139101273 conn(0x7f43100fd830 msgr2=0x7f43100ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.152+0000 7f431761d700 1 -- 172.21.15.171:0/2139101273 shutdown_connections 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 -- 172.21.15.171:0/2139101273 wait complete. 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 Processor -- start 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 -- start start 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f4310075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 0x7f4310074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f431010ded0 con 0x7f4310103490 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f431010dd50 con 0x7f4310103e30 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f431761d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e050 con 0x7f4310102290 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f4315bba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f4315bba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:37936/0 (socket says 172.21.15.171:37936) 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.153+0000 7f4315bba700 1 -- 172.21.15.171:0/2751596978 learned_addr learned my addr 172.21.15.171:0/2751596978 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f43153b9700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f4310075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4314bb8700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 0x7f4310074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 -- 172.21.15.171:0/2751596978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 msgr2=0x7f4310075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f4310075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 -- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 msgr2=0x7f4310074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 0x7f4310074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 -- 172.21.15.171:0/2751596978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4310075410 con 0x7f4310103e30 2024-04-24T22:33:30.158 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f43153b9700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f4310075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4314bb8700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 0x7f4310074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f4315bba700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f430800b970 tx=0x7f430800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308019070 con 0x7f4310103e30 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308014440 con 0x7f4310103e30 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43101a2680 con 0x7f4310103e30 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080075d0 con 0x7f4310103e30 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.154+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a2bc0 con 0x7f4310103e30 2024-04-24T22:33:30.159 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.155+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310068f20 con 0x7f4310103e30 2024-04-24T22:33:30.160 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.156+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f430801c070 con 0x7f4310103e30 2024-04-24T22:33:30.160 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.156+0000 7f43067fc700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:30.160 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.157+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f43080cca30 con 0x7f4310103e30 2024-04-24T22:33:30.160 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.157+0000 7f43153b9700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:30.160 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.158+0000 7f43153b9700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 0x7f42fc0a5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4300005fd0 tx=0x7f4300005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:33:30.163 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.161+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f430809b400 con 0x7f4310103e30 2024-04-24T22:33:30.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:30.366+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4310062670 con 0x7f42fc0a2b90 2024-04-24T22:33:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:30.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:30.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:31.926 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:31 smithi171 ceph-mon[32337]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:31.926 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:31 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi111 2024-04-24T22:33:31.926 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:31 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-24T22:33:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:31 smithi186 ceph-mon[32753]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:31 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi111 2024-04-24T22:33:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:31 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-24T22:33:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:31 smithi111 ceph-mon[27746]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:31 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi111 2024-04-24T22:33:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:31 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-24T22:33:32.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:32 smithi171 ceph-mon[32337]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:32 smithi186 ceph-mon[32753]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:33.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:32 smithi111 ceph-mon[27746]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:33.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:33 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/55576583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:33.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:33 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:33.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:33 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]': finished 2024-04-24T22:33:33.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:33 smithi186 ceph-mon[32753]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T22:33:34.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:33 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:34.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:33 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/55576583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:34.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:33 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:34.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:33 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]': finished 2024-04-24T22:33:34.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:33 smithi111 ceph-mon[27746]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T22:33:34.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:33 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:34.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:33 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/55576583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:34.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:33 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]: dispatch 2024-04-24T22:33:34.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:33 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9336b8fd-028c-49e6-ac02-839e7eb43edf"}]': finished 2024-04-24T22:33:34.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:33 smithi171 ceph-mon[32337]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T22:33:34.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:33 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:34 smithi186 ceph-mon[32753]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:34 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/2486561898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:35.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:34 smithi111 ceph-mon[27746]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:35.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:34 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/2486561898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:35.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:34 smithi171 ceph-mon[32337]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:35.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:34 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/2486561898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:36.943 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:36 smithi171 ceph-mon[32337]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:36.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:36 smithi186 ceph-mon[32753]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:37.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:36 smithi111 ceph-mon[27746]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:38.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:38 smithi171 ceph-mon[32337]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:38 smithi186 ceph-mon[32753]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:38 smithi111 ceph-mon[27746]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:40 smithi186 ceph-mon[32753]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:33:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:41.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:40 smithi111 ceph-mon[27746]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:41.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:33:41.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:41.080 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:40 smithi171 ceph-mon[32337]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:41.080 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:33:41.080 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:41 smithi186 ceph-mon[32753]: Deploying daemon osd.1 on smithi171 2024-04-24T22:33:42.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:41 smithi111 ceph-mon[27746]: Deploying daemon osd.1 on smithi171 2024-04-24T22:33:42.068 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:41 smithi171 ceph-mon[32337]: Deploying daemon osd.1 on smithi171 2024-04-24T22:33:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:42 smithi186 ceph-mon[32753]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:43.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:42 smithi111 ceph-mon[27746]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:43.120 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:42 smithi171 ceph-mon[32337]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:44.859 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:44 smithi171 ceph-mon[32337]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:44 smithi186 ceph-mon[32753]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:45.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:44 smithi111 ceph-mon[27746]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:46 smithi171 ceph-mon[32337]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:46 smithi186 ceph-mon[32753]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:47.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:46 smithi111 ceph-mon[27746]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:49.046 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:49.363 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.361+0000 7f43067fc700 1 -- 172.21.15.171:0/2751596978 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4310062670 con 0x7f42fc0a2b90 2024-04-24T22:33:49.364 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 1 on host 'smithi171' 2024-04-24T22:33:49.365 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 msgr2=0x7f42fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 0x7f42fc0a5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4300005fd0 tx=0x7f4300005f00 comp rx=0 tx=0).stop 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 msgr2=0x7f4310074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f430800b970 tx=0x7f430800bd30 comp rx=0 tx=0).stop 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 shutdown_connections 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:49.366 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4310103e30 0x7f4310074b60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:49.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4310102290 0x7f4310075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:49.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 --2- 172.21.15.171:0/2751596978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310103490 0x7f4310074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:49.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 >> 172.21.15.171:0/2751596978 conn(0x7f43100fd830 msgr2=0x7f4310106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:33:49.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 shutdown_connections 2024-04-24T22:33:49.367 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:33:49.363+0000 7f431761d700 1 -- 172.21.15.171:0/2751596978 wait complete. 2024-04-24T22:33:49.919 DEBUG:teuthology.orchestra.run.smithi171:osd.1> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T22:33:49.921 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-04-24T22:33:49.921 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T22:33:50.163 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:33:50.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:50.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:50.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:50.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:50 smithi186 ceph-mon[32753]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:50.650 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.650 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:50.650 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.650 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:50.651 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:50.651 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:50 smithi171 ceph-mon[32337]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:33:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:50 smithi111 ceph-mon[27746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:33:51.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:51 smithi186 ceph-mon[32753]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:33:51.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:51 smithi186 ceph-mon[32753]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:51.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:51 smithi186 ceph-mon[32753]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T22:33:51.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:51.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:51 smithi186 ceph-mon[32753]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:51.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:51 smithi171 ceph-mon[32337]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:33:51.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:51 smithi171 ceph-mon[32337]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:51.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:51 smithi171 ceph-mon[32337]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T22:33:51.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:51.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:51 smithi171 ceph-mon[32337]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:51.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:51 smithi111 ceph-mon[27746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:33:51.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:51 smithi111 ceph-mon[27746]: from='osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:51.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:51 smithi111 ceph-mon[27746]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T22:33:51.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:51.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:51 smithi111 ceph-mon[27746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131865 s, 795 MB/s 2024-04-24T22:33:52.168 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: from='osd.1 ' entity='osd.1' 2024-04-24T22:33:52.484 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.485 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1[36971]: 2024-04-24T22:33:52.228+0000 7f1364904700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T22:33:52.485 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:33:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1[36971]: 2024-04-24T22:33:52.238+0000 7f135cef0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:33:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:33:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T22:33:52.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:52.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: from='osd.1 ' entity='osd.1' 2024-04-24T22:33:52.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:52 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: from='osd.1 ' entity='osd.1' 2024-04-24T22:33:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:52 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:52.817 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-04-24T22:33:53.068 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:33:53.448 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:53.449 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:53.450 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.450 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] boot 2024-04-24T22:33:53.450 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T22:33:53.450 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:53.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] boot 2024-04-24T22:33:53.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T22:33:53.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:53.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:33:53.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:33:53.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: osd.1 [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] boot 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T22:33:53.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:33:54.649 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:54 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi171 2024-04-24T22:33:54.649 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:54 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-24T22:33:54.649 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:54 smithi186 ceph-mon[32753]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 -- 172.21.15.186:0/3894038875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 msgr2=0x7f7ed41095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3894038875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41095e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00b0a0 tx=0x7f7ecc00b3b0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 -- 172.21.15.186:0/3894038875 shutdown_connections 2024-04-24T22:33:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3894038875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41095e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3894038875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 0x7f7ed41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3894038875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed4102390 0x7f7ed41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 -- 172.21.15.186:0/3894038875 >> 172.21.15.186:0/3894038875 conn(0x7f7ed40fc730 msgr2=0x7f7ed40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 -- 172.21.15.186:0/3894038875 shutdown_connections 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 -- 172.21.15.186:0/3894038875 wait complete. 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.668+0000 7f7edb1de700 1 Processor -- start 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 -- start start 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 0x7f7ed419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed4102390 0x7f7ed419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed410ce00 con 0x7f7ed4102390 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed410cc80 con 0x7f7ed4101190 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed410cf80 con 0x7f7ed4102d30 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:54.673 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:54506/0 (socket says 172.21.15.186:54506) 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 -- 172.21.15.186:0/3783733972 learned_addr learned my addr 172.21.15.186:0/3783733972 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed8f7a700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 0x7f7ed419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 -- 172.21.15.186:0/3783733972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 msgr2=0x7f7ed419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 0x7f7ed419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 -- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed4102390 msgr2=0x7f7ed419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed4102390 0x7f7ed419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 -- 172.21.15.186:0/3783733972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed41a2310 con 0x7f7ed4102d30 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed977b700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009800 tx=0x7f7ecc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc021070 con 0x7f7ed4102d30 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc005520 con 0x7f7ed4102d30 2024-04-24T22:33:54.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed41a25a0 con 0x7f7ed4102d30 2024-04-24T22:33:54.675 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.669+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc01b8d0 con 0x7f7ed4102d30 2024-04-24T22:33:54.675 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.670+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed41a2a70 con 0x7f7ed4102d30 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.670+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed4068f20 con 0x7f7ed4102d30 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.672+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f7ecc024070 con 0x7f7ed4102d30 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.672+0000 7f7ed1ffb700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 0x7f7ebc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.672+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f7ecc00f020 con 0x7f7ed4102d30 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.673+0000 7f7ed8f7a700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 0x7f7ebc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:33:54.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.674+0000 7f7ed8f7a700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 0x7f7ebc0a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ed41021f0 tx=0x7f7ec400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:33:54.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.676+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ecc02b020 con 0x7f7ed4102d30 2024-04-24T22:33:54.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:54 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi171 2024-04-24T22:33:54.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:54 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-24T22:33:54.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:54 smithi111 ceph-mon[27746]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:54.883 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:33:54.881+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7ed40fe850 con 0x7f7ebc0a2b90 2024-04-24T22:33:54.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:54 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi171 2024-04-24T22:33:54.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:54 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-24T22:33:54.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:54 smithi171 ceph-mon[32337]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:55.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:55 smithi186 ceph-mon[32753]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T22:33:55.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:55.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:55.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:55.787 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:55 smithi111 ceph-mon[27746]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T22:33:55.787 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:55.787 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:55.787 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:55.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:55 smithi171 ceph-mon[32337]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T22:33:55.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:33:55.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:33:55.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:33:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:56 smithi186 ceph-mon[32753]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:56 smithi186 ceph-mon[32753]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:56 smithi111 ceph-mon[27746]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:56 smithi111 ceph-mon[27746]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:56 smithi171 ceph-mon[32337]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:33:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:56 smithi171 ceph-mon[32337]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/1626869877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]': finished 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:33:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:33:58 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/1238708080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/1626869877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]': finished 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:33:58.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:33:58 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/1238708080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:33:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:33:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/1626869877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]: dispatch 2024-04-24T22:33:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43993a4c-fb63-47d7-af12-b6cc7f7c2d9b"}]': finished 2024-04-24T22:33:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T22:33:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:33:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:33:58 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/1238708080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:00 smithi171 ceph-mon[32337]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:00 smithi186 ceph-mon[32753]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:01.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:00 smithi111 ceph-mon[27746]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:02.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:02 smithi111 ceph-mon[27746]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:02.856 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:02 smithi186 ceph-mon[32753]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:02.903 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:02 smithi171 ceph-mon[32337]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:04.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:04 smithi186 ceph-mon[32753]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:04.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:04 smithi171 ceph-mon[32337]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:05.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:04 smithi111 ceph-mon[27746]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:05.834 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:05 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:34:05.835 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:05 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:05.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:05 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:34:05.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:05 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:05.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:05 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:34:05.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:05 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:06.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:06 smithi186 ceph-mon[32753]: Deploying daemon osd.2 on smithi186 2024-04-24T22:34:06.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:06 smithi186 ceph-mon[32753]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:06 smithi171 ceph-mon[32337]: Deploying daemon osd.2 on smithi186 2024-04-24T22:34:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:06 smithi171 ceph-mon[32337]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:07.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:06 smithi111 ceph-mon[27746]: Deploying daemon osd.2 on smithi186 2024-04-24T22:34:07.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:06 smithi111 ceph-mon[27746]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:08 smithi171 ceph-mon[32337]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:08.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:08 smithi186 ceph-mon[32753]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:09.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:08 smithi111 ceph-mon[27746]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:10 smithi171 ceph-mon[32337]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:10 smithi186 ceph-mon[32753]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:11.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:10 smithi111 ceph-mon[27746]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:11 smithi171 ceph-mon[32337]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:12.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:11 smithi186 ceph-mon[32753]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:12.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:12.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:12.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:11 smithi111 ceph-mon[27746]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:14.625 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.623+0000 7f7ed1ffb700 1 -- 172.21.15.186:0/3783733972 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ed40fe850 con 0x7f7ebc0a2b90 2024-04-24T22:34:14.625 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-04-24T22:34:14.625 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:14.625 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.625 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.625 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:14.626 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:14.626 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.627 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 msgr2=0x7f7ebc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:14.627 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 0x7f7ebc0a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ed41021f0 tx=0x7f7ec400b410 comp rx=0 tx=0).stop 2024-04-24T22:34:14.627 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 msgr2=0x7f7ed41a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:14.627 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009800 tx=0x7f7ecc00bd70 comp rx=0 tx=0).stop 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 shutdown_connections 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ed4102d30 0x7f7ed41a1c70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ebc0a2b90 0x7f7ebc0a5050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ed4101190 0x7f7ed419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 --2- 172.21.15.186:0/3783733972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed4102390 0x7f7ed419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.625+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 >> 172.21.15.186:0/3783733972 conn(0x7f7ed40fc730 msgr2=0x7f7ed40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.626+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 shutdown_connections 2024-04-24T22:34:14.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:34:14.626+0000 7f7edb1de700 1 -- 172.21.15.186:0/3783733972 wait complete. 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:14.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.332 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T22:34:15.335 INFO:tasks.cephadm:Deploying osd.3 on smithi111 with /dev/vg_nvme/lv_3... 2024-04-24T22:34:15.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T22:34:15.776 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:34:15.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:15.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:15 smithi111 ceph-mon[27746]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:15.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:15 smithi111 ceph-mon[27746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:15.816 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.816 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:15.816 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.816 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:15 smithi186 ceph-mon[32753]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:15.816 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:15 smithi186 ceph-mon[32753]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:15.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:15.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:15.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:15 smithi171 ceph-mon[32337]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:15.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:15 smithi171 ceph-mon[32337]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:34:16.713 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:16.713 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:34:16.713 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:16.714 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T22:34:16.714 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:16.714 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:16 smithi186 ceph-mon[32753]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:16 smithi171 ceph-mon[32337]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:17.030 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:17.031 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:34:17.031 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: from='osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:17.031 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T22:34:17.031 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.031 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:16 smithi111 ceph-mon[27746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:34:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-mon[32753]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:34:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-mon[32753]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T22:34:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-mon[32753]: from='osd.2 ' entity='osd.2' 2024-04-24T22:34:17.720 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2[37937]: 2024-04-24T22:34:17.430+0000 7f8ff38c5700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T22:34:17.720 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2[37937]: 2024-04-24T22:34:17.436+0000 7f8fea6ae700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:34:17.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:17 smithi111 ceph-mon[27746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:34:17.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:17 smithi111 ceph-mon[27746]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T22:34:17.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:17 smithi111 ceph-mon[27746]: from='osd.2 ' entity='osd.2' 2024-04-24T22:34:17.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:17 smithi171 ceph-mon[32337]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:34:17.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:17 smithi171 ceph-mon[32337]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T22:34:17.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:17.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:17 smithi171 ceph-mon[32337]: from='osd.2 ' entity='osd.2' 2024-04-24T22:34:18.630 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T22:34:18.630 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T22:34:18.631 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-24T22:34:18.631 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-24T22:34:18.631 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0381028 s, 275 MB/s 2024-04-24T22:34:18.631 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-24T22:34:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:34:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:34:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] boot 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T22:34:18.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:18.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:18.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] boot 2024-04-24T22:34:18.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T22:34:18.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:34:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:34:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T22:34:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: osd.2 [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] boot 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T22:34:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:34:19.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-04-24T22:34:19.861 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:34:20.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:20 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi186 2024-04-24T22:34:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:20 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-24T22:34:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:20 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi186 2024-04-24T22:34:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:20 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-24T22:34:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:20 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi186 2024-04-24T22:34:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:20 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-24T22:34:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:21.399 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:21 smithi111 ceph-mon[27746]: pgmap v94: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:21.400 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:21 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:21.400 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:21 smithi111 ceph-mon[27746]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T22:34:21.400 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:21 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:21.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:21 smithi171 ceph-mon[32337]: pgmap v94: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:21.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:21 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:21.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:21 smithi171 ceph-mon[32337]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T22:34:21.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:21 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:21.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:21 smithi186 ceph-mon[32753]: pgmap v94: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:21.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:21 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:21.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:21 smithi186 ceph-mon[32753]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T22:34:21.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:21 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:34:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 -- 172.21.15.111:0/3796111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 msgr2=0x7f93f41026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 --2- 172.21.15.111:0/3796111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f41026c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f93e40098f0 tx=0x7f93e4009c00 comp rx=0 tx=0).stop 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 -- 172.21.15.111:0/3796111704 shutdown_connections 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 --2- 172.21.15.111:0/3796111704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f4103e40 0x7f93f410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 --2- 172.21.15.111:0/3796111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f41026c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 --2- 172.21.15.111:0/3796111704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f41034a0 0x7f93f4103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 -- 172.21.15.111:0/3796111704 >> 172.21.15.111:0/3796111704 conn(0x7f93f40fd800 msgr2=0x7f93f40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 -- 172.21.15.111:0/3796111704 shutdown_connections 2024-04-24T22:34:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.883+0000 7f93fa400700 1 -- 172.21.15.111:0/3796111704 wait complete. 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.884+0000 7f93fa400700 1 Processor -- start 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 -- start start 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f41034a0 0x7f93f4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f4103e40 0x7f93f419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410de90 con 0x7f93f4103e40 2024-04-24T22:34:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410dd10 con 0x7f93f41034a0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93fa400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410e010 con 0x7f93f41022a0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93f3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93f3e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f4103e40 0x7f93f419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93f3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:39746/0 (socket says 172.21.15.111:39746) 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93f3400700 1 -- 172.21.15.111:0/3555426918 learned_addr learned my addr 172.21.15.111:0/3555426918 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.885+0000 7f93f2a00700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f41034a0 0x7f93f4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 -- 172.21.15.111:0/3555426918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f41034a0 msgr2=0x7f93f4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f41034a0 0x7f93f4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 -- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f4103e40 msgr2=0x7f93f419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f4103e40 0x7f93f419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 -- 172.21.15.111:0/3555426918 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f41a0160 con 0x7f93f41022a0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93f3400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009230 tx=0x7f93e40052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400b560 con 0x7f93f41022a0 2024-04-24T22:34:21.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f41a03f0 con 0x7f93f41022a0 2024-04-24T22:34:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e400bbe0 con 0x7f93f41022a0 2024-04-24T22:34:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.886+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4017400 con 0x7f93f41022a0 2024-04-24T22:34:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.887+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f41a08a0 con 0x7f93f41022a0 2024-04-24T22:34:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.888+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f4068f20 con 0x7f93f41022a0 2024-04-24T22:34:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.890+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f93e4003680 con 0x7f93f41022a0 2024-04-24T22:34:21.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.891+0000 7f93ebe00700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 0x7f93d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:21.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.891+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f93e40ce9d0 con 0x7f93f41022a0 2024-04-24T22:34:21.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.891+0000 7f93f2a00700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 0x7f93d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:21.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.892+0000 7f93f2a00700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 0x7f93d80a50d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005fd0 tx=0x7f93e0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:34:21.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:21.896+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f93e4013030 con 0x7f93f41022a0 2024-04-24T22:34:22.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:21 smithi111 sudo[47478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:22.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:21 smithi111 sudo[47478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:22.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:21 smithi111 sudo[47478]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:22.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:21 smithi111 sudo[47478]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:22.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:22.129+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f93f4062670 con 0x7f93d80a2c10 2024-04-24T22:34:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:22.180+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f93e4098d40 con 0x7f93f41022a0 2024-04-24T22:34:22.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:22.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T22:34:22.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:22.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:22.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:22.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:22.511 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:34:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T22:34:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:22.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:22.783 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:22 smithi111 sudo[47548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:22.783 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:22 smithi111 sudo[47548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:22.783 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:22 smithi111 sudo[47548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:22.784 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:34:22 smithi111 sudo[47548]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:23.062 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:22 smithi171 sudo[40780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:23.063 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:22 smithi171 sudo[40780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:23.063 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:22 smithi171 sudo[40780]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:23.063 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:22 smithi171 sudo[40780]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:23.358 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:23 smithi111 ceph-mon[27746]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:23.358 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:23 smithi111 ceph-mon[27746]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T22:34:23.358 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:23 smithi111 ceph-mon[27746]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T22:34:23.358 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.358 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.406 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:23 smithi171 sudo[40783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:23.406 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:23 smithi171 sudo[40783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:23.406 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:23 smithi171 sudo[40783]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:23.406 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:34:23 smithi171 sudo[40783]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:23.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:23 smithi171 ceph-mon[32337]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:23.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:23 smithi171 ceph-mon[32337]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T22:34:23.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:23 smithi171 ceph-mon[32337]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T22:34:23.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.499 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42018]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:23.499 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:23.499 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42018]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:23.499 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42018]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:23.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:23 smithi186 ceph-mon[32753]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:23.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:23 smithi186 ceph-mon[32753]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T22:34:23.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:23 smithi186 ceph-mon[32753]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T22:34:23.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:23.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:23.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:23.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:23.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:34:23 smithi186 sudo[42021]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:24.479 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:24.479 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:24.479 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:24.479 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47626]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:25.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47652]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:25.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47652]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47652]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:24 smithi111 sudo[47652]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:25.286 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:25.287 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:25.287 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:25.287 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42024]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:25.547 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:25.547 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:25.547 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:25.547 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:34:25.548 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:25.548 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40786]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:25.548 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40786]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:25.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42027]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:25.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42027]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:25.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42027]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:25.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 sudo[42027]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:25.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:25.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:25.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:25 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:25.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:25.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:25 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:25.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:25 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:34:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:34:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40789]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:34:25.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:25 smithi171 sudo[40789]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:34:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3749366951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]': finished 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.658 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3749366951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]': finished 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T22:34:26.659 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3749366951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]: dispatch 2024-04-24T22:34:26.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7cb3cc-81a9-4f99-802b-d29c399eb6b1"}]': finished 2024-04-24T22:34:26.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T22:34:26.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:27.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:27 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1891397285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:27.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:27 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1891397285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:27.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:27 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1891397285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:28 smithi171 ceph-mon[32337]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:28.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:28 smithi186 ceph-mon[32753]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:28.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:28 smithi111 ceph-mon[27746]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:30 smithi171 ceph-mon[32337]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T22:34:30.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:30 smithi111 ceph-mon[27746]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T22:34:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:30 smithi186 ceph-mon[32753]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T22:34:32.801 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:32 smithi186 ceph-mon[32753]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-04-24T22:34:32.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:32 smithi171 ceph-mon[32337]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-04-24T22:34:33.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:32 smithi111 ceph-mon[27746]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-04-24T22:34:34.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:34 smithi111 ceph-mon[27746]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:34.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:34 smithi171 ceph-mon[32337]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:34.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:34 smithi186 ceph-mon[32753]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:35.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:34:35.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:35.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:34:35.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:36.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:34:36.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:36.596 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:36 smithi111 ceph-mon[27746]: Deploying daemon osd.3 on smithi111 2024-04-24T22:34:36.596 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:36 smithi111 ceph-mon[27746]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:36.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:36 smithi171 ceph-mon[32337]: Deploying daemon osd.3 on smithi111 2024-04-24T22:34:36.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:36 smithi171 ceph-mon[32337]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:36.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:36 smithi186 ceph-mon[32753]: Deploying daemon osd.3 on smithi111 2024-04-24T22:34:36.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:36 smithi186 ceph-mon[32753]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:34:38.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:38 smithi171 ceph-mon[32337]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-04-24T22:34:39.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:38 smithi186 ceph-mon[32753]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-04-24T22:34:39.000 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:38 smithi111 ceph-mon[27746]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-04-24T22:34:40.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:40 smithi111 ceph-mon[27746]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T22:34:40.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:40 smithi171 ceph-mon[32337]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T22:34:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:40 smithi186 ceph-mon[32753]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T22:34:42.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:42 smithi171 ceph-mon[32337]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:42 smithi186 ceph-mon[32753]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:43.011 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:42 smithi111 ceph-mon[27746]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:44 smithi186 ceph-mon[32753]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:44.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:44.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:44 smithi111 ceph-mon[27746]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:44.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:44 smithi171 ceph-mon[32337]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:44.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:46.670 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:46 smithi111 ceph-mon[27746]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:46.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:46 smithi171 ceph-mon[32337]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:46 smithi186 ceph-mon[32753]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:47.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:47 smithi171 ceph-mon[32337]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:34:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:47 smithi186 ceph-mon[32753]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:47 smithi111 ceph-mon[27746]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:34:48.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.479+0000 7f93ebe00700 1 -- 172.21.15.111:0/3555426918 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f93f4062670 con 0x7f93d80a2c10 2024-04-24T22:34:48.482 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 3 on host 'smithi111' 2024-04-24T22:34:48.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 msgr2=0x7f93d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:48.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 0x7f93d80a50d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005fd0 tx=0x7f93e0009310 comp rx=0 tx=0).stop 2024-04-24T22:34:48.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 msgr2=0x7f93f4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:48.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009230 tx=0x7f93e40052f0 comp rx=0 tx=0).stop 2024-04-24T22:34:48.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 shutdown_connections 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93f4103e40 0x7f93f419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f41022a0 0x7f93f4198500 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f93d80a2c10 0x7f93d80a50d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 --2- 172.21.15.111:0/3555426918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93f41034a0 0x7f93f4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.483+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 >> 172.21.15.111:0/3555426918 conn(0x7f93f40fd800 msgr2=0x7f93f4106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.484+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 shutdown_connections 2024-04-24T22:34:48.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:34:48.484+0000 7f93fa400700 1 -- 172.21.15.111:0/3555426918 wait complete. 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:48 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:49.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:48 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:49.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:49.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:48 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:34:49.232 DEBUG:teuthology.orchestra.run.smithi111:osd.3> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T22:34:49.238 INFO:tasks.cephadm:Deploying osd.4 on smithi171 with /dev/vg_nvme/lv_3... 2024-04-24T22:34:49.238 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T22:34:49.483 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:34:49.903 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:49 smithi171 ceph-mon[32337]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:34:49.904 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:49 smithi171 ceph-mon[32337]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T22:34:49.904 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:49 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.904 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:49 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.914 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:49 smithi111 ceph-mon[27746]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:34:49.914 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:49 smithi111 ceph-mon[27746]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T22:34:49.914 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:49 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.914 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:49 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:49 smithi186 ceph-mon[32753]: from='osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:34:49.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:49 smithi186 ceph-mon[32753]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T22:34:49.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:49 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:49.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:49 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:50.849 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:34:50.849 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:34:50.849 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:50.850 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] boot 2024-04-24T22:34:50.850 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T22:34:50.850 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:50.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:34:50.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:34:50.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:50.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] boot 2024-04-24T22:34:50.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T22:34:50.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: osd.3 [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] boot 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T22:34:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112861 s, 929 MB/s 2024-04-24T22:34:51.498 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-24T22:34:51.983 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:51 smithi111 ceph-mon[27746]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T22:34:51.983 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:51 smithi111 ceph-mon[27746]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T22:34:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:51 smithi186 ceph-mon[32753]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T22:34:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:51 smithi186 ceph-mon[32753]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T22:34:52.066 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:51 smithi171 ceph-mon[32337]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T22:34:52.067 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:51 smithi171 ceph-mon[32337]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T22:34:52.096 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-04-24T22:34:52.342 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:34:52.797 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:52 smithi171 ceph-mon[32337]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:34:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:52 smithi186 ceph-mon[32753]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:34:53.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:52 smithi111 ceph-mon[27746]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:34:53.787 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:53 smithi171 ceph-mon[32337]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T22:34:53.787 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:53 smithi171 ceph-mon[32337]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T22:34:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:53 smithi186 ceph-mon[32753]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T22:34:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:53 smithi186 ceph-mon[32753]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- 172.21.15.171:0/1529190444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 msgr2=0x7f76bc105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 --2- 172.21.15.171:0/1529190444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc105120 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009990 tx=0x7f76ac009ca0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- 172.21.15.171:0/1529190444 shutdown_connections 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 --2- 172.21.15.171:0/1529190444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 0x7f76bc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 --2- 172.21.15.171:0/1529190444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc105120 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 --2- 172.21.15.171:0/1529190444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76bc105660 0x7f76bc107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- 172.21.15.171:0/1529190444 >> 172.21.15.171:0/1529190444 conn(0x7f76bc0fc760 msgr2=0x7f76bc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:34:54.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- 172.21.15.171:0/1529190444 shutdown_connections 2024-04-24T22:34:54.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- 172.21.15.171:0/1529190444 wait complete. 2024-04-24T22:34:54.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 Processor -- start 2024-04-24T22:34:54.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.027+0000 7f76c1955700 1 -- start start 2024-04-24T22:34:54.031 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:54.031 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76bc105660 0x7f76bc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 0x7f76bc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc10df40 con 0x7f76bc105660 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc10ddc0 con 0x7f76bc102d30 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c1955700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc10e0c0 con 0x7f76bc107f90 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:50558/0 (socket says 172.21.15.171:50558) 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 -- 172.21.15.171:0/1722509705 learned_addr learned my addr 172.21.15.171:0/1722509705 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76bb7fe700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 0x7f76bc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 -- 172.21.15.171:0/1722509705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 msgr2=0x7f76bc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 0x7f76bc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 -- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76bc105660 msgr2=0x7f76bc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:34:54.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76bc105660 0x7f76bc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 -- 172.21.15.171:0/1722509705 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1a01b0 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76baffd700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b5c0 tx=0x7f76ac009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac027070 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.028+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac004150 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.029+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1a0440 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.029+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00f460 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.029+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1a0970 con 0x7f76bc102d30 2024-04-24T22:34:54.033 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.029+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc068f20 con 0x7f76bc102d30 2024-04-24T22:34:54.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:53 smithi111 ceph-mon[27746]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T22:34:54.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:53 smithi111 ceph-mon[27746]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T22:34:54.034 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.031+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f76ac02a070 con 0x7f76bc102d30 2024-04-24T22:34:54.034 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.031+0000 7f76c0953700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 0x7f76a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:34:54.034 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.031+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f76ac0cf460 con 0x7f76bc102d30 2024-04-24T22:34:54.035 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.032+0000 7f76b27fc700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 0x7f76a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:34:54.035 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.032+0000 7f76b27fc700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 0x7f76a80a5160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76a40097b0 tx=0x7f76a4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:34:54.036 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.035+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76ac031020 con 0x7f76bc102d30 2024-04-24T22:34:54.240 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:34:54.238+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f76bc062670 con 0x7f76a80a2ca0 2024-04-24T22:34:54.996 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:54 smithi171 ceph-mon[32337]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T22:34:54.996 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:54.996 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:54.996 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:54 smithi186 ceph-mon[32753]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T22:34:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:54 smithi111 ceph-mon[27746]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T22:34:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:34:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:34:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:55.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:55 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.006 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:56.006 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.006 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.006 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.006 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.007 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.007 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:56.007 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:56.007 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:55 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:34:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:34:56.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:34:56.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:55 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:34:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:56 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi111 2024-04-24T22:34:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:56 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-24T22:34:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:56 smithi186 ceph-mon[32753]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-24T22:34:57.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:56 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi111 2024-04-24T22:34:57.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:56 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-24T22:34:57.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:56 smithi111 ceph-mon[27746]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-24T22:34:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:56 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi111 2024-04-24T22:34:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:56 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-24T22:34:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:56 smithi171 ceph-mon[32337]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-24T22:34:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/2293275326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]': finished 2024-04-24T22:34:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T22:34:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:34:58.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:57 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/204965232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:58.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/2293275326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]': finished 2024-04-24T22:34:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T22:34:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:34:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:57 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/204965232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/2293275326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]: dispatch 2024-04-24T22:34:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d8ef87e-1192-4e17-8861-c2214d514b34"}]': finished 2024-04-24T22:34:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T22:34:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:34:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:57 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/204965232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:34:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:34:58 smithi186 ceph-mon[32753]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-24T22:34:59.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:34:58 smithi111 ceph-mon[27746]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-24T22:34:59.161 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:34:58 smithi171 ceph-mon[32337]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-24T22:35:01.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:00 smithi111 ceph-mon[27746]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T22:35:01.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:00 smithi171 ceph-mon[32337]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T22:35:01.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:00 smithi186 ceph-mon[32753]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T22:35:03.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:02 smithi111 ceph-mon[27746]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:03.051 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:02 smithi186 ceph-mon[32753]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:03.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:02 smithi171 ceph-mon[32337]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:05.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:04 smithi111 ceph-mon[27746]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:35:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:05.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:04 smithi171 ceph-mon[32337]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:05.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:35:05.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:05.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:04 smithi186 ceph-mon[32753]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:05.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:35:05.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:06.018 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:05 smithi171 ceph-mon[32337]: Deploying daemon osd.4 on smithi171 2024-04-24T22:35:06.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:05 smithi111 ceph-mon[27746]: Deploying daemon osd.4 on smithi171 2024-04-24T22:35:06.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:05 smithi186 ceph-mon[32753]: Deploying daemon osd.4 on smithi171 2024-04-24T22:35:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:06 smithi171 ceph-mon[32337]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:07.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:06 smithi111 ceph-mon[27746]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:07.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:06 smithi186 ceph-mon[32753]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:07 smithi171 ceph-mon[32337]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:08.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:07 smithi111 ceph-mon[27746]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:08.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:07 smithi186 ceph-mon[32753]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:10.633 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:10 smithi171 ceph-mon[32337]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:10.633 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:10 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:10.633 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:10 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:10.633 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:10 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:10 smithi186 ceph-mon[32753]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:10 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:10 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:10 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:11.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:10 smithi111 ceph-mon[27746]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:11.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:10 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:11.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:10 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:11.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:10 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:12.631 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:12 smithi171 ceph-mon[32337]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:12 smithi186 ceph-mon[32753]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:13.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:12 smithi111 ceph-mon[27746]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:13.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.669+0000 7f76c0953700 1 -- 172.21.15.171:0/1722509705 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76bc062670 con 0x7f76a80a2ca0 2024-04-24T22:35:13.672 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 4 on host 'smithi171' 2024-04-24T22:35:13.673 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 msgr2=0x7f76a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 0x7f76a80a5160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76a40097b0 tx=0x7f76a4006d20 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 msgr2=0x7f76bc198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b5c0 tx=0x7f76ac009f90 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 shutdown_connections 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc107f90 0x7f76bc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76a80a2ca0 0x7f76a80a5160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76bc102d30 0x7f76bc198550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 --2- 172.21.15.171:0/1722509705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76bc105660 0x7f76bc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 >> 172.21.15.171:0/1722509705 conn(0x7f76bc0fc760 msgr2=0x7f76bc103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 shutdown_connections 2024-04-24T22:35:13.674 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:35:13.672+0000 7f76c1955700 1 -- 172.21.15.171:0/1722509705 wait complete. 2024-04-24T22:35:13.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:13.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:13.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:13 smithi171 ceph-mon[32337]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:13.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:13.852 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:13 smithi111 ceph-mon[27746]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:14.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:13 smithi186 ceph-mon[32753]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:14.440 DEBUG:teuthology.orchestra.run.smithi171:osd.4> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T22:35:14.442 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-04-24T22:35:14.442 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T22:35:14.685 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:35:15.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:14 smithi186 ceph-mon[32753]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:15.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:14 smithi186 ceph-mon[32753]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:15.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:14 smithi171 ceph-mon[32337]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:15.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:14 smithi171 ceph-mon[32337]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:15.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:14 smithi111 ceph-mon[27746]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:15.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:14 smithi111 ceph-mon[27746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:16.042 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:15 smithi186 ceph-mon[32753]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:35:16.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:15 smithi171 ceph-mon[32337]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:35:16.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: from='osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:15 smithi111 ceph-mon[27746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115177 s, 910 MB/s 2024-04-24T22:35:16.639 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-24T22:35:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-mon[32337]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T22:35:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-mon[32337]: from='osd.4 ' entity='osd.4' 2024-04-24T22:35:16.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:16.907 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:35:16.621+0000 7f286304b700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T22:35:16.907 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:35:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:35:16.631+0000 7f285b637700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:35:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:16 smithi186 ceph-mon[32753]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T22:35:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:16 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:16 smithi186 ceph-mon[32753]: from='osd.4 ' entity='osd.4' 2024-04-24T22:35:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:16 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:17.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:16 smithi111 ceph-mon[27746]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T22:35:17.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:16 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:17.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:16 smithi111 ceph-mon[27746]: from='osd.4 ' entity='osd.4' 2024-04-24T22:35:17.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:16 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:17.312 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-04-24T22:35:17.564 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi171 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:17.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] boot 2024-04-24T22:35:17.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T22:35:17.934 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi171 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:18.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] boot 2024-04-24T22:35:18.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T22:35:18.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi171 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: osd.4 [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] boot 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T22:35:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:35:18.921 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:18.922 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:18.922 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:18.922 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:18.922 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:18 smithi186 ceph-mon[32753]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T22:35:19.143 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.140+0000 7fedf4862700 1 -- 172.21.15.186:0/3696345252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0103490 msgr2=0x7fedf01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.140+0000 7fedf4862700 1 --2- 172.21.15.186:0/3696345252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0103490 0x7fedf01038f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fede0009990 tx=0x7fede0009ca0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 -- 172.21.15.186:0/3696345252 shutdown_connections 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 --2- 172.21.15.186:0/3696345252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 0x7fedf010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 --2- 172.21.15.186:0/3696345252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedf0102290 0x7fedf01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 --2- 172.21.15.186:0/3696345252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0103490 0x7fedf01038f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 -- 172.21.15.186:0/3696345252 >> 172.21.15.186:0/3696345252 conn(0x7fedf00fd830 msgr2=0x7fedf00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 -- 172.21.15.186:0/3696345252 shutdown_connections 2024-04-24T22:35:19.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.141+0000 7fedf4862700 1 -- 172.21.15.186:0/3696345252 wait complete. 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 Processor -- start 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 -- start start 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedf0103490 0x7fedf0074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 0x7fedf0074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010df10 con 0x7fedf0103490 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010dd90 con 0x7fedf0103e30 2024-04-24T22:35:19.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedf4862700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010e090 con 0x7fedf0102290 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:52100/0 (socket says 172.21.15.186:52100) 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 -- 172.21.15.186:0/2178144189 learned_addr learned my addr 172.21.15.186:0/2178144189 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedeed9d700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 0x7fedf0074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 -- 172.21.15.186:0/2178144189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 msgr2=0x7fedf0074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 0x7fedf0074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 -- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedf0103490 msgr2=0x7fedf0074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedf0103490 0x7fedf0074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 -- 172.21.15.186:0/2178144189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0075410 con 0x7fedf0102290 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fedee59c700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b500 tx=0x7fedd800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.142+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8015020 con 0x7fedf0102290 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.143+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01a26e0 con 0x7fedf0102290 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.143+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8011d10 con 0x7fedf0102290 2024-04-24T22:35:19.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.143+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8010e50 con 0x7fedf0102290 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.143+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a2c70 con 0x7fedf0102290 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.143+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0068f20 con 0x7fedf0102290 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.145+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fedd8018020 con 0x7fedf0102290 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.145+0000 7fede77fe700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 0x7feddc0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.146+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fedd80cdcd0 con 0x7fedf0102290 2024-04-24T22:35:19.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.146+0000 7fededd9b700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 0x7feddc0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:19.149 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.147+0000 7fededd9b700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 0x7feddc0a5280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fede0005f50 tx=0x7fede0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:19.152 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.150+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedd801f020 con 0x7fedf0102290 2024-04-24T22:35:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:18 smithi171 ceph-mon[32337]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T22:35:19.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:19.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:19.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:19.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:19.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:18 smithi111 ceph-mon[27746]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T22:35:19.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:19.356+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fedf0062670 con 0x7feddc0a2dc0 2024-04-24T22:35:20.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:19 smithi171 ceph-mon[32337]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:20.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:20.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:20.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:20.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:19 smithi171 ceph-mon[32337]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:20.167 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:19 smithi186 ceph-mon[32753]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:20.167 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:20.167 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:20.167 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:20.167 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:19 smithi186 ceph-mon[32753]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:20.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:19 smithi111 ceph-mon[27746]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:20.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:19 smithi111 ceph-mon[27746]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/1319516029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:22.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]': finished 2024-04-24T22:35:22.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T22:35:22.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:22.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:22 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/2183018062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/1319516029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]': finished 2024-04-24T22:35:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T22:35:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:22 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/2183018062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/1319516029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]: dispatch 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac00518-3f56-44e1-84fb-9f5f57fb12a3"}]': finished 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:22 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/2183018062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:24.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:24 smithi171 ceph-mon[32337]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:24 smithi186 ceph-mon[32753]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:25.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:24 smithi111 ceph-mon[27746]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:26 smithi186 ceph-mon[32753]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:26.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:26 smithi171 ceph-mon[32337]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:26 smithi111 ceph-mon[27746]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:28 smithi171 ceph-mon[32337]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:28 smithi186 ceph-mon[32753]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:28 smithi111 ceph-mon[27746]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:35:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:29.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:35:29.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:35:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:30.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:30 smithi186 ceph-mon[32753]: Deploying daemon osd.5 on smithi186 2024-04-24T22:35:30.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:30 smithi186 ceph-mon[32753]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:30 smithi171 ceph-mon[32337]: Deploying daemon osd.5 on smithi186 2024-04-24T22:35:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:30 smithi171 ceph-mon[32337]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:30.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:30 smithi111 ceph-mon[27746]: Deploying daemon osd.5 on smithi186 2024-04-24T22:35:30.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:30 smithi111 ceph-mon[27746]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:32.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:32 smithi171 ceph-mon[32337]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:32 smithi186 ceph-mon[32753]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:33.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:32 smithi111 ceph-mon[27746]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:34.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:34 smithi186 ceph-mon[32753]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:34.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:34 smithi171 ceph-mon[32337]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:35.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:34 smithi111 ceph-mon[27746]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:35 smithi186 ceph-mon[32753]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:36.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:35 smithi111 ceph-mon[27746]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:36.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:36.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:35 smithi171 ceph-mon[32337]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:38.655 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:38.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:38.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:38.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:38.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:38.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:38.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:38.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:38.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:38.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:38.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:38.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.043 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-04-24T22:35:39.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.041+0000 7fede77fe700 1 -- 172.21.15.186:0/2178144189 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fedf0062670 con 0x7feddc0a2dc0 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 msgr2=0x7feddc0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 0x7feddc0a5280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fede0005f50 tx=0x7fede0005e80 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 msgr2=0x7fedf0075fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b500 tx=0x7fedd800b810 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 shutdown_connections 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7feddc0a2dc0 0x7feddc0a5280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0103e30 0x7fedf0074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf0102290 0x7fedf0075fd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 --2- 172.21.15.186:0/2178144189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedf0103490 0x7fedf0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 >> 172.21.15.186:0/2178144189 conn(0x7fedf00fd830 msgr2=0x7fedf01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 shutdown_connections 2024-04-24T22:35:39.046 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:35:39.044+0000 7fedf4862700 1 -- 172.21.15.186:0/2178144189 wait complete. 2024-04-24T22:35:39.676 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:39 smithi186 ceph-mon[32753]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.677 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:39 smithi186 ceph-mon[32753]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.677 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.677 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.677 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:39.705 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T22:35:39.707 INFO:tasks.cephadm:Deploying osd.6 on smithi111 with /dev/vg_nvme/lv_2... 2024-04-24T22:35:39.707 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T22:35:39.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:39 smithi111 ceph-mon[27746]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:39 smithi111 ceph-mon[27746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:39.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:39 smithi171 ceph-mon[32337]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:39 smithi171 ceph-mon[32337]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:35:39.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:39.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:40.165 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:35:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:35:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T22:35:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:40.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-mon[32753]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:40.751 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:35:40.582+0000 7f83d83cd700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T22:35:40.751 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:35:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:35:40.589+0000 7f83d49c1700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:35:40.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:40.886 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:35:40.886 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:40.886 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T22:35:40.886 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:40.886 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:40 smithi111 ceph-mon[27746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:40.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:40.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:35:40.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: from='osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:40.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T22:35:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:40 smithi171 ceph-mon[32337]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] boot 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T22:35:41.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] boot 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T22:35:41.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:42.026 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:35:42.026 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T22:35:42.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:42.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: osd.5 [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] boot 2024-04-24T22:35:42.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T22:35:42.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:35:42.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:35:42.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:35:42.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T22:35:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T22:35:43.269 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-24T22:35:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-24T22:35:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330509 s, 317 MB/s 2024-04-24T22:35:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi186 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:43.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:43 smithi111 ceph-mon[27746]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T22:35:43.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi186 2024-04-24T22:35:43.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-24T22:35:43.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:43.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:43.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:43.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:43 smithi171 ceph-mon[32337]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi186 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:35:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:43 smithi186 ceph-mon[32753]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T22:35:44.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-04-24T22:35:44.644 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:35:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:44 smithi171 ceph-mon[32337]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:44 smithi171 ceph-mon[32337]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T22:35:44.968 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:44 smithi111 ceph-mon[27746]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:44.968 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:44 smithi111 ceph-mon[27746]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T22:35:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:44 smithi186 ceph-mon[32753]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:44 smithi186 ceph-mon[32753]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T22:35:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:45 smithi171 ceph-mon[32337]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T22:35:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:45 smithi171 ceph-mon[32337]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:45 smithi186 ceph-mon[32753]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T22:35:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:45 smithi186 ceph-mon[32753]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:46.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:45 smithi111 ceph-mon[27746]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T22:35:46.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:45 smithi111 ceph-mon[27746]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-24T22:35:46.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 -- 172.21.15.111:0/3292563336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102d20 msgr2=0x7f49dc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 --2- 172.21.15.111:0/3292563336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102d20 0x7f49dc1095d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009fc0 tx=0x7f49d400b330 comp rx=0 tx=0).stop 2024-04-24T22:35:46.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 -- 172.21.15.111:0/3292563336 shutdown_connections 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 --2- 172.21.15.111:0/3292563336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102d20 0x7f49dc1095d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 --2- 172.21.15.111:0/3292563336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 --2- 172.21.15.111:0/3292563336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102380 0x7f49dc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 -- 172.21.15.111:0/3292563336 >> 172.21.15.111:0/3292563336 conn(0x7f49dc0fc720 msgr2=0x7f49dc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 -- 172.21.15.111:0/3292563336 shutdown_connections 2024-04-24T22:35:46.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.852+0000 7f49e1800700 1 -- 172.21.15.111:0/3292563336 wait complete. 2024-04-24T22:35:46.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.853+0000 7f49e1800700 1 Processor -- start 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.853+0000 7f49e1800700 1 -- start start 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 0x7f49dc19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10cdd0 con 0x7f49dc102d20 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10cc50 con 0x7f49dc102380 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49e1800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10cf50 con 0x7f49dc101180 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49daa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49db400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 0x7f49dc19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49da000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49daa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:59220/0 (socket says 172.21.15.111:59220) 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49da000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:49956/0 (socket says 172.21.15.111:49956) 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.854+0000 7f49daa00700 1 -- 172.21.15.111:0/2793856616 learned_addr learned my addr 172.21.15.111:0/2793856616 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 -- 172.21.15.111:0/2793856616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 msgr2=0x7f49dc06b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 -- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 msgr2=0x7f49dc19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 0x7f49dc19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 -- 172.21.15.111:0/2793856616 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc1a0110 con 0x7f49dc102380 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49daa00700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49db400700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 0x7f49dc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:35:46.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49da000700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49c800eb90 tx=0x7f49c800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:46.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c800cc90 con 0x7f49dc102380 2024-04-24T22:35:46.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c800ce30 con 0x7f49dc102380 2024-04-24T22:35:46.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc1a0400 con 0x7f49dc102380 2024-04-24T22:35:46.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.855+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8010430 con 0x7f49dc102380 2024-04-24T22:35:46.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.856+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a0940 con 0x7f49dc102380 2024-04-24T22:35:46.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.857+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc068f20 con 0x7f49dc102380 2024-04-24T22:35:46.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.859+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f49c8003680 con 0x7f49dc102380 2024-04-24T22:35:46.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.860+0000 7f49d3400700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 0x7f49c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.861+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f49c80ce520 con 0x7f49dc102380 2024-04-24T22:35:46.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.861+0000 7f49daa00700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 0x7f49c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.861+0000 7f49daa00700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 0x7f49c00a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005950 tx=0x7f49cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:46.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:46.865+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f49c809c2a0 con 0x7f49dc102380 2024-04-24T22:35:47.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:35:47.099+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f49dc062670 con 0x7f49c00a2c20 2024-04-24T22:35:47.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:47.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:47.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:47.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:47.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:47.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:47.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:35:47.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:35:47.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:48.344 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:48 smithi111 ceph-mon[27746]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:48.344 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:48 smithi111 ceph-mon[27746]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:35:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:48 smithi186 ceph-mon[32753]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:48 smithi186 ceph-mon[32753]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:35:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:48 smithi171 ceph-mon[32337]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:48 smithi171 ceph-mon[32337]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:35:50.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:50 smithi171 ceph-mon[32337]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:35:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:50 smithi186 ceph-mon[32753]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:35:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:50 smithi111 ceph-mon[27746]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:35:51.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:51 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]: dispatch 2024-04-24T22:35:51.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:51 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]': finished 2024-04-24T22:35:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:51 smithi171 ceph-mon[32337]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T22:35:51.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:51 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:35:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:51 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]: dispatch 2024-04-24T22:35:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:51 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]': finished 2024-04-24T22:35:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:51 smithi186 ceph-mon[32753]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T22:35:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:51 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:35:52.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:51 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]: dispatch 2024-04-24T22:35:52.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:51 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2115221398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e612d700-d87e-4031-b2f9-6455030a282b"}]': finished 2024-04-24T22:35:52.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:51 smithi111 ceph-mon[27746]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T22:35:52.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:51 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:35:52.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:52 smithi171 ceph-mon[32337]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:52.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:52 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2130926266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:52 smithi186 ceph-mon[32753]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:53.014 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:52 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2130926266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:53.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:52 smithi111 ceph-mon[27746]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:53.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:52 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2130926266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:35:54.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:54 smithi171 ceph-mon[32337]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:54 smithi186 ceph-mon[32753]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:55.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:54 smithi111 ceph-mon[27746]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:56 smithi111 ceph-mon[27746]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:56 smithi171 ceph-mon[32337]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:56 smithi186 ceph-mon[32753]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:58.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:35:58 smithi171 ceph-mon[32337]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:35:58 smithi186 ceph-mon[32753]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:35:59.017 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:35:58 smithi111 ceph-mon[27746]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:00 smithi171 ceph-mon[32337]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:00 smithi186 ceph-mon[32753]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:01.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:00 smithi111 ceph-mon[27746]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:01.859 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T22:36:01.860 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:01.860 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:01 smithi111 ceph-mon[27746]: Deploying daemon osd.6 on smithi111 2024-04-24T22:36:01.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T22:36:01.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:01.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:01 smithi171 ceph-mon[32337]: Deploying daemon osd.6 on smithi111 2024-04-24T22:36:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T22:36:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:01 smithi186 ceph-mon[32753]: Deploying daemon osd.6 on smithi111 2024-04-24T22:36:02.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:02 smithi111 ceph-mon[27746]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:02.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:02 smithi171 ceph-mon[32337]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:02.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:02 smithi186 ceph-mon[32753]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:04.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:04 smithi171 ceph-mon[32337]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:04.955 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:04 smithi111 ceph-mon[27746]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:04.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:04 smithi186 ceph-mon[32753]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:06 smithi171 ceph-mon[32337]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:06 smithi186 ceph-mon[32753]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:07.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:06 smithi111 ceph-mon[27746]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:08.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:08 smithi111 ceph-mon[27746]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:08 smithi171 ceph-mon[32337]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:08 smithi186 ceph-mon[32753]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:09.862 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:09.863 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:09.863 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:10.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:10.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:10.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:10.643 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:10 smithi111 ceph-mon[27746]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:10 smithi171 ceph-mon[32337]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:10 smithi186 ceph-mon[32753]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:12 smithi171 ceph-mon[32337]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:12.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:12 smithi111 ceph-mon[27746]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:12.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:12.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:12 smithi186 ceph-mon[32753]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:13.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.656+0000 7f49d3400700 1 -- 172.21.15.111:0/2793856616 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49dc062670 con 0x7f49c00a2c20 2024-04-24T22:36:13.658 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 6 on host 'smithi111' 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 msgr2=0x7f49c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 0x7f49c00a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005950 tx=0x7f49cc009260 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 msgr2=0x7f49dc19d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49c800eb90 tx=0x7f49c800eea0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 shutdown_connections 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49dc102d20 0x7f49dc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49dc101180 0x7f49dc06b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f49c00a2c20 0x7f49c00a50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 --2- 172.21.15.111:0/2793856616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49dc102380 0x7f49dc19d1b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 >> 172.21.15.111:0/2793856616 conn(0x7f49dc0fc720 msgr2=0x7f49dc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 shutdown_connections 2024-04-24T22:36:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:36:13.659+0000 7f49e1800700 1 -- 172.21.15.111:0/2793856616 wait complete. 2024-04-24T22:36:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:13 smithi171 ceph-mon[32337]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T22:36:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:13 smithi186 ceph-mon[32753]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T22:36:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:14.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:14.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:13 smithi111 ceph-mon[27746]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T22:36:14.028 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:14.028 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:14.514 DEBUG:teuthology.orchestra.run.smithi111:osd.6> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T22:36:14.520 INFO:tasks.cephadm:Deploying osd.7 on smithi171 with /dev/vg_nvme/lv_2... 2024-04-24T22:36:14.520 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T22:36:14.769 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:36:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:14 smithi111 ceph-mon[27746]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:14 smithi111 ceph-mon[27746]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T22:36:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:14 smithi111 ceph-mon[27746]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T22:36:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:14 smithi111 ceph-mon[27746]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:36:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:14 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:14.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:14 smithi171 ceph-mon[32337]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:14.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:14 smithi171 ceph-mon[32337]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T22:36:14.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:14 smithi171 ceph-mon[32337]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T22:36:14.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:14 smithi171 ceph-mon[32337]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:36:14.893 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:14 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:14 smithi186 ceph-mon[32753]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:14 smithi186 ceph-mon[32753]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T22:36:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:14 smithi186 ceph-mon[32753]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T22:36:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:14 smithi186 ceph-mon[32753]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:36:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:14 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.905 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:36:15.719+0000 7f11cb400700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T22:36:15.905 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:36:15.750+0000 7f11bfe00700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:36:15.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:36:15.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:36:15.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:36:15.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T22:36:15.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.934 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113473 s, 924 MB/s 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-04-24T22:36:16.692 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-24T22:36:16.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:16 smithi186 ceph-mon[32753]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:16.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:16 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:16.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:16 smithi186 ceph-mon[32753]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' 2024-04-24T22:36:17.025 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:16 smithi171 ceph-mon[32337]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:17.026 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:16 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:17.026 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:16 smithi171 ceph-mon[32337]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' 2024-04-24T22:36:17.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:16 smithi111 ceph-mon[27746]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:17.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:16 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:17.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:16 smithi111 ceph-mon[27746]: from='osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463]' entity='osd.6' 2024-04-24T22:36:17.330 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-04-24T22:36:17.585 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:36:17.888 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:17.888 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:17 smithi171 ceph-mon[32337]: osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] boot 2024-04-24T22:36:17.888 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:17 smithi171 ceph-mon[32337]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T22:36:17.888 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:17 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:17.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:17.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:17 smithi186 ceph-mon[32753]: osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] boot 2024-04-24T22:36:17.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:17 smithi186 ceph-mon[32753]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T22:36:17.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:17 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:18.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:18.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:17 smithi111 ceph-mon[27746]: osd.6 [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] boot 2024-04-24T22:36:18.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:17 smithi111 ceph-mon[27746]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T22:36:18.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:17 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:36:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:18 smithi111 ceph-mon[27746]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:18 smithi111 ceph-mon[27746]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T22:36:18.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:18 smithi171 ceph-mon[32337]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:18.991 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:18 smithi171 ceph-mon[32337]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T22:36:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:18 smithi186 ceph-mon[32753]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-24T22:36:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:18 smithi186 ceph-mon[32753]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T22:36:19.206 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 -- 172.21.15.171:0/2914026386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102390 msgr2=0x7f3c381027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.206 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/2914026386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102390 0x7f3c381027f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c280099c0 tx=0x7f3c28009cd0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.206 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 -- 172.21.15.171:0/2914026386 shutdown_connections 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/2914026386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38102d30 0x7f3c381095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/2914026386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38101190 0x7f3c381015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/2914026386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102390 0x7f3c381027f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 -- 172.21.15.171:0/2914026386 >> 172.21.15.171:0/2914026386 conn(0x7f3c380fc730 msgr2=0x7f3c380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 -- 172.21.15.171:0/2914026386 shutdown_connections 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.204+0000 7f3c3d20e700 1 -- 172.21.15.171:0/2914026386 wait complete. 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 Processor -- start 2024-04-24T22:36:19.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 -- start start 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38101190 0x7f3c38197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38102390 0x7f3c38197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3810cd80 con 0x7f3c38101190 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3810cc00 con 0x7f3c38102d30 2024-04-24T22:36:19.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3d20e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3810cf00 con 0x7f3c38102390 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:33896/0 (socket says 172.21.15.171:33896) 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3759e700 1 -- 172.21.15.171:0/1502988914 learned_addr learned my addr 172.21.15.171:0/1502988914 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c3659c700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38102390 0x7f3c38197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.205+0000 7f3c36d9d700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38101190 0x7f3c38197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 -- 172.21.15.171:0/1502988914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38102390 msgr2=0x7f3c38197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38102390 0x7f3c38197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 -- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38101190 msgr2=0x7f3c38197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38101190 0x7f3c38197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 -- 172.21.15.171:0/1502988914 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3819f090 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3759e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000ada0 tx=0x7f3c3000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30017020 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3000a0d0 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3819f380 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30005480 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.206+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3819f870 con 0x7f3c38102d30 2024-04-24T22:36:19.213 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.207+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c38068f20 con 0x7f3c38102d30 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.208+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3c30005620 con 0x7f3c38102d30 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.209+0000 7f3c1ffff700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 0x7f3c200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.209+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f3c300ce070 con 0x7f3c38102d30 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.209+0000 7f3c36d9d700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 0x7f3c200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.210+0000 7f3c36d9d700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 0x7f3c200a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005950 tx=0x7f3c2c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:19.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.212+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3c30021020 con 0x7f3c38102d30 2024-04-24T22:36:19.427 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:19.425+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3c38062730 con 0x7f3c200a2c20 2024-04-24T22:36:20.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi111 2024-04-24T22:36:20.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:20.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:20 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi111 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:20 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi111 2024-04-24T22:36:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:20.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:20 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/595834682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:22.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]': finished 2024-04-24T22:36:22.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T22:36:22.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:22 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/595834682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]': finished 2024-04-24T22:36:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T22:36:23.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:22 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/595834682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]: dispatch 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac3e009c-2c29-480c-b226-9df2d6f0d5ed"}]': finished 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T22:36:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:22 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:23.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:23 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/1176989267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:23.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:23 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/1176989267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:24.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:23 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/1176989267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:24.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:24 smithi171 ceph-mon[32337]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:24 smithi186 ceph-mon[32753]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:25.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:24 smithi111 ceph-mon[27746]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:26 smithi111 ceph-mon[27746]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:26.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:26 smithi171 ceph-mon[32337]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:26 smithi186 ceph-mon[32753]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:28 smithi171 ceph-mon[32337]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:28 smithi186 ceph-mon[32753]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:29.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:28 smithi111 ceph-mon[27746]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:29.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T22:36:29.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T22:36:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:30.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T22:36:30.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:30 smithi171 ceph-mon[32337]: Deploying daemon osd.7 on smithi171 2024-04-24T22:36:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:30 smithi171 ceph-mon[32337]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:30.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:30 smithi111 ceph-mon[27746]: Deploying daemon osd.7 on smithi171 2024-04-24T22:36:30.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:30 smithi111 ceph-mon[27746]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:30 smithi186 ceph-mon[32753]: Deploying daemon osd.7 on smithi171 2024-04-24T22:36:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:30 smithi186 ceph-mon[32753]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:32.858 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:32 smithi171 ceph-mon[32337]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:32 smithi186 ceph-mon[32753]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:33.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:32 smithi111 ceph-mon[27746]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:34.744 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:34 smithi171 ceph-mon[32337]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:34 smithi186 ceph-mon[32753]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:35.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:34 smithi111 ceph-mon[27746]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:36.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:35 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:36.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:35 smithi186 ceph-mon[32753]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:36.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:35 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:35 smithi111 ceph-mon[27746]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:35 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:36.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:35 smithi171 ceph-mon[32337]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:38.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.765+0000 7f3c1ffff700 1 -- 172.21.15.171:0/1502988914 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c38062730 con 0x7f3c200a2c20 2024-04-24T22:36:38.767 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 7 on host 'smithi171' 2024-04-24T22:36:38.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 msgr2=0x7f3c200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:38.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 0x7f3c200a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005950 tx=0x7f3c2c0058e0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 msgr2=0x7f3c3819e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:38.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000ada0 tx=0x7f3c3000c330 comp rx=0 tx=0).stop 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 shutdown_connections 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3c200a2c20 0x7f3c200a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c38102d30 0x7f3c3819e9f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c38101190 0x7f3c38197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 --2- 172.21.15.171:0/1502988914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c38102390 0x7f3c38197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.767+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 >> 172.21.15.171:0/1502988914 conn(0x7f3c380fc730 msgr2=0x7f3c38105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.768+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 shutdown_connections 2024-04-24T22:36:38.770 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:36:38.768+0000 7f3c3d20e700 1 -- 172.21.15.171:0/1502988914 wait complete. 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.388 DEBUG:teuthology.orchestra.run.smithi171:osd.7> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T22:36:39.393 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-04-24T22:36:39.393 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T22:36:39.638 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:36:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:39 smithi186 ceph-mon[32753]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:39 smithi186 ceph-mon[32753]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:39 smithi111 ceph-mon[27746]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:39 smithi111 ceph-mon[27746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:40.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:40.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:39 smithi171 ceph-mon[32337]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:40.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:39 smithi171 ceph-mon[32337]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:40.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:40 smithi171 ceph-mon[32337]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T22:36:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T22:36:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:41.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:40 smithi186 ceph-mon[32753]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:41.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:41.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T22:36:41.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: from='osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:41.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T22:36:41.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:41.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:40 smithi111 ceph-mon[27746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:36:41.585 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T22:36:41.585 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T22:36:41.586 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-24T22:36:41.586 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-24T22:36:41.586 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130105 s, 806 MB/s 2024-04-24T22:36:41.586 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: from='osd.7 ' entity='osd.7' 2024-04-24T22:36:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-mon[32337]: osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] boot 2024-04-24T22:36:41.907 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:36:41.593+0000 7ffa1bd72700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T22:36:41.908 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:36:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:36:41.607+0000 7ffa1435e700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: from='osd.7 ' entity='osd.7' 2024-04-24T22:36:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:41 smithi111 ceph-mon[27746]: osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] boot 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: from='osd.7 ' entity='osd.7' 2024-04-24T22:36:42.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:41 smithi186 ceph-mon[32753]: osd.7 [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] boot 2024-04-24T22:36:42.320 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-04-24T22:36:42.558 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:36:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:36:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:36:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:43.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:42 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:43.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:42 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:43.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:36:43.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:42 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:36:43.934 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:43 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi171 2024-04-24T22:36:43.934 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:43 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-24T22:36:43.934 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:43 smithi186 ceph-mon[32753]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T22:36:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:43 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi171 2024-04-24T22:36:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:43 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-24T22:36:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:43 smithi111 ceph-mon[27746]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 -- 172.21.15.186:0/1496385104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888105710 msgr2=0x7f8888107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 --2- 172.21.15.186:0/1496385104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888105710 0x7f8888107b00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8878009990 tx=0x7f8878009ca0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 -- 172.21.15.186:0/1496385104 shutdown_connections 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 --2- 172.21.15.186:0/1496385104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888108040 0x7f888810a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 --2- 172.21.15.186:0/1496385104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888102de0 0x7f88881051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 --2- 172.21.15.186:0/1496385104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888105710 0x7f8888107b00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 -- 172.21.15.186:0/1496385104 >> 172.21.15.186:0/1496385104 conn(0x7f88880fc790 msgr2=0x7f88880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 -- 172.21.15.186:0/1496385104 shutdown_connections 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.120+0000 7f88900d7700 1 -- 172.21.15.186:0/1496385104 wait complete. 2024-04-24T22:36:44.124 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 Processor -- start 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 -- start start 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888102de0 0x7f88881984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888105710 0x7f8888198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f888810df40 con 0x7f8888105710 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f888810ddc0 con 0x7f8888102de0 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f88900d7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f888810e0c0 con 0x7f8888108040 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:54728/0 (socket says 172.21.15.186:54728) 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 -- 172.21.15.186:0/1890135616 learned_addr learned my addr 172.21.15.186:0/1890135616 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888de73700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888102de0 0x7f88881984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 -- 172.21.15.186:0/1890135616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888102de0 msgr2=0x7f88881984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:44.125 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888102de0 0x7f88881984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 -- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888105710 msgr2=0x7f8888198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888105710 0x7f8888198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 -- 172.21.15.186:0/1890135616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88881a0130 con 0x7f8888108040 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.121+0000 7f888e674700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8880007ee0 tx=0x7f888000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.122+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880004db0 con 0x7f8888108040 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.122+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888000dbd0 con 0x7f8888108040 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.122+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88881a0420 con 0x7f8888108040 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.122+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880015400 con 0x7f8888108040 2024-04-24T22:36:44.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.122+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a0a30 con 0x7f8888108040 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.123+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8888068f20 con 0x7f8888108040 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.124+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8880005000 con 0x7f8888108040 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.124+0000 7f887effd700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 0x7f88740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.124+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f88800cdec0 con 0x7f8888108040 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.125+0000 7f888de73700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 0x7f88740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:44.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.126+0000 7f888de73700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 0x7f88740a5160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8884009a10 tx=0x7f8884009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:44.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.129+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8880010070 con 0x7f8888108040 2024-04-24T22:36:44.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:43 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi171 2024-04-24T22:36:44.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:43 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-24T22:36:44.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:43 smithi171 ceph-mon[32337]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T22:36:44.334 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:36:44.333+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8888062670 con 0x7f88740a2ca0 2024-04-24T22:36:45.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:44 smithi111 ceph-mon[27746]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:44 smithi111 ceph-mon[27746]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T22:36:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:45.058 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:44 smithi186 ceph-mon[32753]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:45.058 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:44 smithi186 ceph-mon[32753]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T22:36:45.058 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:45.058 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:45.058 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:45.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:44 smithi171 ceph-mon[32337]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:45.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:44 smithi171 ceph-mon[32337]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T22:36:45.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:36:45.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:36:45.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:46.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:45 smithi111 ceph-mon[27746]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:45 smithi111 ceph-mon[27746]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T22:36:46.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:45 smithi186 ceph-mon[32753]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:46.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:45 smithi186 ceph-mon[32753]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T22:36:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:45 smithi171 ceph-mon[32337]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:45 smithi171 ceph-mon[32337]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T22:36:47.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:46 smithi111 ceph-mon[27746]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:46 smithi111 ceph-mon[27746]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T22:36:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:46 smithi171 ceph-mon[32337]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:47.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:46 smithi171 ceph-mon[32337]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T22:36:47.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:46 smithi186 ceph-mon[32753]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 368 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:47.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:46 smithi186 ceph-mon[32753]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/3327513404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]': finished 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:36:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/906660283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:48.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/3327513404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]': finished 2024-04-24T22:36:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T22:36:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:36:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/906660283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/3327513404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]: dispatch 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "789c1edf-d22f-4a42-9b25-02cedbccadd4"}]': finished 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:36:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/906660283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:36:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:48 smithi171 ceph-mon[32337]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:48 smithi171 ceph-mon[32337]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T22:36:49.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:48 smithi171 ceph-mon[32337]: Cluster is now healthy 2024-04-24T22:36:49.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:48 smithi186 ceph-mon[32753]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:49.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:48 smithi186 ceph-mon[32753]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T22:36:49.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:48 smithi186 ceph-mon[32753]: Cluster is now healthy 2024-04-24T22:36:49.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:48 smithi111 ceph-mon[27746]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:49.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:48 smithi111 ceph-mon[27746]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T22:36:49.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:48 smithi111 ceph-mon[27746]: Cluster is now healthy 2024-04-24T22:36:50.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:49 smithi171 ceph-mon[32337]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:50.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:49 smithi186 ceph-mon[32753]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:50.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:49 smithi111 ceph-mon[27746]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:52.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:52 smithi171 ceph-mon[32337]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:52 smithi186 ceph-mon[32753]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:53.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:52 smithi111 ceph-mon[27746]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:54.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:54 smithi171 ceph-mon[32337]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:54.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T22:36:54.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:54 smithi186 ceph-mon[32753]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T22:36:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:55.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:54 smithi111 ceph-mon[27746]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:55.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T22:36:55.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:55.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:55 smithi186 ceph-mon[32753]: Deploying daemon osd.8 on smithi186 2024-04-24T22:36:55.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:55 smithi171 ceph-mon[32337]: Deploying daemon osd.8 on smithi186 2024-04-24T22:36:56.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:55 smithi111 ceph-mon[27746]: Deploying daemon osd.8 on smithi186 2024-04-24T22:36:56.697 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:56 smithi186 ceph-mon[32753]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:56 smithi171 ceph-mon[32337]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:57.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:56 smithi111 ceph-mon[27746]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:58.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:36:58 smithi171 ceph-mon[32337]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:36:58 smithi186 ceph-mon[32753]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:36:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:36:58 smithi111 ceph-mon[27746]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:00.655 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:00 smithi186 ceph-mon[32753]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:00.655 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.655 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.656 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:00.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:00 smithi111 ceph-mon[27746]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:00.872 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.872 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.872 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:00 smithi171 ceph-mon[32337]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:02.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:02 smithi111 ceph-mon[27746]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:02.905 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:02 smithi171 ceph-mon[32337]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:02.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:02 smithi186 ceph-mon[32753]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:03.368 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.365+0000 7f887effd700 1 -- 172.21.15.186:0/1890135616 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8888062670 con 0x7f88740a2ca0 2024-04-24T22:37:03.368 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 msgr2=0x7f88740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 0x7f88740a5160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8884009a10 tx=0x7f8884009190 comp rx=0 tx=0).stop 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 msgr2=0x7f888819fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8880007ee0 tx=0x7f888000d3b0 comp rx=0 tx=0).stop 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 shutdown_connections 2024-04-24T22:37:03.370 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888108040 0x7f888819fa90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f88740a2ca0 0x7f88740a5160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888102de0 0x7f88881984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 --2- 172.21.15.186:0/1890135616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888105710 0x7f8888198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 >> 172.21.15.186:0/1890135616 conn(0x7f88880fc790 msgr2=0x7f8888103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 shutdown_connections 2024-04-24T22:37:03.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:37:03.368+0000 7f88900d7700 1 -- 172.21.15.186:0/1890135616 wait complete. 2024-04-24T22:37:04.056 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T22:37:04.059 INFO:tasks.cephadm:Deploying osd.9 on smithi111 with /dev/vg_nvme/lv_1... 2024-04-24T22:37:04.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T22:37:04.492 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:37:04.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T22:37:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:04.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:05.647 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:05 smithi111 ceph-mon[27746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T22:37:05.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:05 smithi111 ceph-mon[27746]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T22:37:05.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:05 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:05.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:05 smithi111 ceph-mon[27746]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:05.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:05 smithi111 ceph-mon[27746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:05.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:05 smithi171 ceph-mon[32337]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T22:37:05.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:05 smithi171 ceph-mon[32337]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T22:37:05.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:05 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:05.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:05 smithi171 ceph-mon[32337]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:05.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:05 smithi171 ceph-mon[32337]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:05.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:05 smithi186 ceph-mon[32753]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T22:37:05.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:05 smithi186 ceph-mon[32753]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T22:37:05.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:05 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:05.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:05 smithi186 ceph-mon[32753]: from='osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:05.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:05 smithi186 ceph-mon[32753]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:37:06.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-mon[32753]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:37:06.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-mon[32753]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T22:37:06.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:06.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-mon[32753]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:06.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-mon[32753]: from='osd.8 ' entity='osd.8' 2024-04-24T22:37:06.500 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:37:06.172+0000 7fc3271d9700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T22:37:06.500 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:37:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:37:06.189+0000 7fc31f7c5700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:37:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:06 smithi171 ceph-mon[32337]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:37:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:06 smithi171 ceph-mon[32337]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T22:37:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:06 smithi171 ceph-mon[32337]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:06 smithi171 ceph-mon[32337]: from='osd.8 ' entity='osd.8' 2024-04-24T22:37:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:06 smithi111 ceph-mon[27746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:37:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:06 smithi111 ceph-mon[27746]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T22:37:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:06 smithi111 ceph-mon[27746]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T22:37:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:06 smithi111 ceph-mon[27746]: from='osd.8 ' entity='osd.8' 2024-04-24T22:37:07.335 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T22:37:07.335 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T22:37:07.335 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-24T22:37:07.335 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-24T22:37:07.336 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353094 s, 297 MB/s 2024-04-24T22:37:07.336 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-24T22:37:07.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:37:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:37:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] boot 2024-04-24T22:37:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T22:37:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:07 smithi171 ceph-mon[32337]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] boot 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:07.678 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:07 smithi186 ceph-mon[32753]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: osd.8 [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] boot 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:37:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:07 smithi111 ceph-mon[27746]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T22:37:08.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-04-24T22:37:08.781 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:37:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi186 2024-04-24T22:37:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi186 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi186 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T22:37:09.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:09.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:09.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:09.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:09.971 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:09 smithi111 ceph-mon[27746]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T22:37:09.972 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:09 smithi111 ceph-mon[27746]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:10.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:09 smithi171 ceph-mon[32337]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T22:37:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:09 smithi171 ceph-mon[32337]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:10.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:09 smithi186 ceph-mon[32753]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T22:37:10.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:09 smithi186 ceph-mon[32753]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 681 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:10.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 -- 172.21.15.111:0/716091195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 msgr2=0x7f39bc110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:10.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 --2- 172.21.15.111:0/716091195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc110d20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39b400b0d0 tx=0x7f39b400b3e0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 -- 172.21.15.111:0/716091195 shutdown_connections 2024-04-24T22:37:10.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 --2- 172.21.15.111:0/716091195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc110d20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 --2- 172.21.15.111:0/716091195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc1088d0 0x7f39bc108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 --2- 172.21.15.111:0/716091195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc109ad0 0x7f39bc109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 -- 172.21.15.111:0/716091195 >> 172.21.15.111:0/716091195 conn(0x7f39bc0778a0 msgr2=0x7f39bc079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:10.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 -- 172.21.15.111:0/716091195 shutdown_connections 2024-04-24T22:37:10.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.856+0000 7f39c3400700 1 -- 172.21.15.111:0/716091195 wait complete. 2024-04-24T22:37:10.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.857+0000 7f39c3400700 1 Processor -- start 2024-04-24T22:37:10.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.857+0000 7f39c3400700 1 -- start start 2024-04-24T22:37:10.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 0x7f39bc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:10.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 0x7f39bc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc1a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1144c0 con 0x7f39bc10a470 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc114340 con 0x7f39bc1088d0 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c3400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc114640 con 0x7f39bc109ad0 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c1400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc1a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c0a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 0x7f39bc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 0x7f39bc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c0a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 0x7f39bc19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34762/0 (socket says 172.21.15.111:34762) 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c0a00700 1 -- 172.21.15.111:0/3356405598 learned_addr learned my addr 172.21.15.111:0/3356405598 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 -- 172.21.15.111:0/3356405598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 msgr2=0x7f39bc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 0x7f39bc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 -- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 msgr2=0x7f39bc1a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:10.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc1a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:10.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39bbe00700 1 -- 172.21.15.111:0/3356405598 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a6720 con 0x7f39bc109ad0 2024-04-24T22:37:10.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.858+0000 7f39c1400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc1a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:37:10.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39bbe00700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 0x7f39bc19f000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a800ea20 tx=0x7f39a800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:10.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a800cbd0 con 0x7f39bc109ad0 2024-04-24T22:37:10.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a800ee60 con 0x7f39bc109ad0 2024-04-24T22:37:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a6a10 con 0x7f39bc109ad0 2024-04-24T22:37:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8018680 con 0x7f39bc109ad0 2024-04-24T22:37:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.859+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a6f50 con 0x7f39bc109ad0 2024-04-24T22:37:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.860+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc068f20 con 0x7f39bc109ad0 2024-04-24T22:37:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.862+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f39a8018820 con 0x7f39bc109ad0 2024-04-24T22:37:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.864+0000 7f39b9600700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 0x7f39a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.864+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f39a80ce7c0 con 0x7f39bc109ad0 2024-04-24T22:37:10.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.864+0000 7f39c0a00700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 0x7f39a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:10.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.865+0000 7f39c0a00700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 0x7f39a40a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005950 tx=0x7f39b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:10.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:10.868+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39a809c030 con 0x7f39bc109ad0 2024-04-24T22:37:11.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:10 smithi111 ceph-mon[27746]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T22:37:11.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:11.107+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f39bc062670 con 0x7f39a40a2c20 2024-04-24T22:37:11.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:10 smithi171 ceph-mon[32337]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T22:37:11.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:10 smithi186 ceph-mon[32753]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T22:37:11.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:11 smithi111 ceph-mon[27746]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:11.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:11.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:11.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:11.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:11 smithi111 ceph-mon[27746]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:11 smithi171 ceph-mon[32337]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:11 smithi171 ceph-mon[32337]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:11 smithi186 ceph-mon[32753]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:11 smithi186 ceph-mon[32753]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:14.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:14 smithi171 ceph-mon[32337]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T22:37:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:14 smithi186 ceph-mon[32753]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T22:37:15.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:14 smithi111 ceph-mon[27746]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T22:37:15.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:15 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1744896019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:15.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:15 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:15.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:15 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]': finished 2024-04-24T22:37:15.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:15 smithi171 ceph-mon[32337]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T22:37:15.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:15 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1744896019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:15 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:15 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]': finished 2024-04-24T22:37:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:15 smithi186 ceph-mon[32753]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T22:37:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:16.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:15 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1744896019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:16.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:15 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]: dispatch 2024-04-24T22:37:16.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:15 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa3d5a06-570a-4e45-874e-c64a64f66b9b"}]': finished 2024-04-24T22:37:16.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:15 smithi111 ceph-mon[27746]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T22:37:16.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:16 smithi186 ceph-mon[32753]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:37:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:16 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2602180484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:16 smithi171 ceph-mon[32337]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:37:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:16 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2602180484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:17.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:16 smithi111 ceph-mon[27746]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:37:17.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:16 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2602180484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:18.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:18 smithi171 ceph-mon[32337]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:37:18.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:18 smithi186 ceph-mon[32753]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:37:19.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:18 smithi111 ceph-mon[27746]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:37:20.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:20 smithi171 ceph-mon[32337]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:37:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:20 smithi186 ceph-mon[32753]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:37:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:20 smithi111 ceph-mon[27746]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:37:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:22 smithi171 ceph-mon[32337]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T22:37:22.993 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:22 smithi111 ceph-mon[27746]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T22:37:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:22 smithi186 ceph-mon[32753]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T22:37:24.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:24 smithi171 ceph-mon[32337]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:24.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:24 smithi111 ceph-mon[27746]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:24 smithi186 ceph-mon[32753]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T22:37:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:25 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:25.980 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T22:37:25.980 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:25 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:25.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T22:37:25.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:25 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:26.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:26 smithi171 ceph-mon[32337]: Deploying daemon osd.9 on smithi111 2024-04-24T22:37:26.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:26 smithi171 ceph-mon[32337]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:26 smithi186 ceph-mon[32753]: Deploying daemon osd.9 on smithi111 2024-04-24T22:37:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:26 smithi186 ceph-mon[32753]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:27.002 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:26 smithi111 ceph-mon[27746]: Deploying daemon osd.9 on smithi111 2024-04-24T22:37:27.002 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:26 smithi111 ceph-mon[27746]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:28.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:28 smithi171 ceph-mon[32337]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:28 smithi186 ceph-mon[32753]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:29.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:28 smithi111 ceph-mon[27746]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:30 smithi111 ceph-mon[27746]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:30 smithi171 ceph-mon[32337]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:30 smithi186 ceph-mon[32753]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:32.765 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:32 smithi111 ceph-mon[27746]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:32.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:32 smithi171 ceph-mon[32337]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:33.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:32 smithi186 ceph-mon[32753]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:34.517 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.517 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.517 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:34.517 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:34 smithi111 ceph-mon[27746]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:34.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:34.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:34 smithi171 ceph-mon[32337]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:34 smithi186 ceph-mon[32753]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:36.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:36 smithi171 ceph-mon[32337]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:36.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:36 smithi186 ceph-mon[32753]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:37.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:36 smithi111 ceph-mon[27746]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.212+0000 7f39b9600700 1 -- 172.21.15.111:0/3356405598 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39bc062670 con 0x7f39a40a2c20 2024-04-24T22:37:38.214 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 9 on host 'smithi111' 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 msgr2=0x7f39a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 0x7f39a40a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005950 tx=0x7f39b0009260 comp rx=0 tx=0).stop 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 msgr2=0x7f39bc19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 0x7f39bc19f000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a800ea20 tx=0x7f39a800ed30 comp rx=0 tx=0).stop 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 shutdown_connections 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39bc10a470 0x7f39bc1a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:38.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39a40a2c20 0x7f39a40a50e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:38.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39bc1088d0 0x7f39bc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:38.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 --2- 172.21.15.111:0/3356405598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc109ad0 0x7f39bc19f000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:38.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.215+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 >> 172.21.15.111:0/3356405598 conn(0x7f39bc0778a0 msgr2=0x7f39bc10cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:38.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.216+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 shutdown_connections 2024-04-24T22:37:38.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:37:38.216+0000 7f39c3400700 1 -- 172.21.15.111:0/3356405598 wait complete. 2024-04-24T22:37:38.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T22:37:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:38 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T22:37:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:38 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T22:37:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:38.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:38 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:38.940 DEBUG:teuthology.orchestra.run.smithi111:osd.9> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T22:37:38.947 INFO:tasks.cephadm:Deploying osd.10 on smithi171 with /dev/vg_nvme/lv_1... 2024-04-24T22:37:38.947 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T22:37:39.194 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:37:39.538 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:39 smithi171 ceph-mon[32337]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T22:37:39.538 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:39 smithi171 ceph-mon[32337]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T22:37:39.538 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:39 smithi171 ceph-mon[32337]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:37:39.538 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:39 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:39.604 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-mon[27746]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T22:37:39.605 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-mon[27746]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T22:37:39.605 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-mon[27746]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:37:39.605 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:39.605 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:37:39.305+0000 7fa1ec200700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T22:37:39.605 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:37:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:37:39.323+0000 7fa1e7a00700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:37:39.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:39 smithi186 ceph-mon[32753]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T22:37:39.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:39 smithi186 ceph-mon[32753]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T22:37:39.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:39 smithi186 ceph-mon[32753]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-24T22:37:39.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:39 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.642 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:40 smithi171 ceph-mon[32337]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:37:40.642 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:40 smithi171 ceph-mon[32337]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T22:37:40.642 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.642 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:40 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.642 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:40 smithi171 ceph-mon[32337]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:40.656 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:40 smithi111 ceph-mon[27746]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:37:40.656 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:40 smithi111 ceph-mon[27746]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T22:37:40.656 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.656 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:40 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.656 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:40 smithi111 ceph-mon[27746]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:40 smithi186 ceph-mon[32753]: from='osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-24T22:37:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:40 smithi186 ceph-mon[32753]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T22:37:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:40 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:40 smithi186 ceph-mon[32753]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111624 s, 939 MB/s 2024-04-24T22:37:41.205 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-24T22:37:41.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:41 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:37:41.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:41 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:37:41.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:41 smithi171 ceph-mon[32337]: osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] boot 2024-04-24T22:37:41.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:41 smithi171 ceph-mon[32337]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T22:37:41.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:41 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:41.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:41 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:37:41.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:41 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:37:41.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:41 smithi186 ceph-mon[32753]: osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] boot 2024-04-24T22:37:41.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:41 smithi186 ceph-mon[32753]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T22:37:41.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:41 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:41 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:37:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:41 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:37:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:41 smithi111 ceph-mon[27746]: osd.9 [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] boot 2024-04-24T22:37:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:41 smithi111 ceph-mon[27746]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T22:37:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:41 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:37:41.805 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-04-24T22:37:42.040 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:37:42.480 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:42 smithi171 ceph-mon[32337]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T22:37:42.480 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:42 smithi171 ceph-mon[32337]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:42 smithi186 ceph-mon[32753]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T22:37:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:42 smithi186 ceph-mon[32753]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:42.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:42 smithi111 ceph-mon[27746]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T22:37:42.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:42 smithi111 ceph-mon[27746]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 -- 172.21.15.171:0/3739715162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c101190 msgr2=0x7f3e0c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 --2- 172.21.15.171:0/3739715162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c101190 0x7f3e0c1015b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0099c0 tx=0x7f3dfc009cd0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 -- 172.21.15.171:0/3739715162 shutdown_connections 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 --2- 172.21.15.171:0/3739715162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c102d30 0x7f3e0c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 --2- 172.21.15.171:0/3739715162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c101190 0x7f3e0c1015b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 --2- 172.21.15.171:0/3739715162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c102390 0x7f3e0c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.595 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 -- 172.21.15.171:0/3739715162 >> 172.21.15.171:0/3739715162 conn(0x7f3e0c0fc750 msgr2=0x7f3e0c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 -- 172.21.15.171:0/3739715162 shutdown_connections 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.591+0000 7f3e1238c700 1 -- 172.21.15.171:0/3739715162 wait complete. 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 Processor -- start 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 -- start start 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c101190 0x7f3e0c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c102390 0x7f3e0c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c10cdf0 con 0x7f3e0c102390 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c10cc70 con 0x7f3e0c102d30 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e1238c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c10cf70 con 0x7f3e0c101190 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e10929700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e10929700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:33426/0 (socket says 172.21.15.171:33426) 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e10929700 1 -- 172.21.15.171:0/1808429410 learned_addr learned my addr 172.21.15.171:0/1808429410 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-24T22:37:43.596 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.592+0000 7f3e0bfff700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c101190 0x7f3e0c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 -- 172.21.15.171:0/1808429410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c101190 msgr2=0x7f3e0c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c101190 0x7f3e0c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 -- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c102390 msgr2=0x7f3e0c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c102390 0x7f3e0c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 -- 172.21.15.171:0/1808429410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c1a0160 con 0x7f3e0c102d30 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e10929700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400ef30 tx=0x7f3e0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040092e0 con 0x7f3e0c102d30 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c1a03f0 con 0x7f3e0c102d30 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e04004bb0 con 0x7f3e0c102d30 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04004280 con 0x7f3e0c102d30 2024-04-24T22:37:43.597 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.593+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c1a0a00 con 0x7f3e0c102d30 2024-04-24T22:37:43.598 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.594+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c068f20 con 0x7f3e0c102d30 2024-04-24T22:37:43.598 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.595+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3e0401b030 con 0x7f3e0c102d30 2024-04-24T22:37:43.598 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.596+0000 7f3e097fa700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 0x7f3df40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.598 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.596+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f3e040ce2a0 con 0x7f3e0c102d30 2024-04-24T22:37:43.598 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.596+0000 7f3e0bfff700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 0x7f3df40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:43.599 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.597+0000 7f3e0bfff700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 0x7f3df40a5160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b580 tx=0x7f3dfc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:43.602 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.600+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e04022020 con 0x7f3e0c102d30 2024-04-24T22:37:43.811 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:37:43.809+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3e0c0fe910 con 0x7f3df40a2ca0 2024-04-24T22:37:44.706 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:44.706 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:44.707 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:44.708 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:44.708 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:44 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:45.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:45.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:44 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:37:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:37:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:45.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:44 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:37:45.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:45 smithi186 ceph-mon[32753]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:45.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:45 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi111 2024-04-24T22:37:45.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:45 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-24T22:37:45.845 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:45 smithi171 ceph-mon[32337]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:45.845 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:45 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi111 2024-04-24T22:37:45.845 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:45 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-24T22:37:46.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:45 smithi111 ceph-mon[27746]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:45 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi111 2024-04-24T22:37:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:45 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-24T22:37:46.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:46 smithi171 ceph-mon[32337]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:46 smithi186 ceph-mon[32753]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:47.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:46 smithi111 ceph-mon[27746]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:47.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/3095365979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]': finished 2024-04-24T22:37:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T22:37:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:37:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.171:0/1092950123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/3095365979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]': finished 2024-04-24T22:37:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T22:37:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:37:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.171:0/1092950123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/3095365979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]: dispatch 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13df39cd-031b-431a-90a9-76d1c09f1922"}]': finished 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:37:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.171:0/1092950123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:37:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:48 smithi186 ceph-mon[32753]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:49.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:48 smithi111 ceph-mon[27746]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:49.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:48 smithi171 ceph-mon[32337]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:50.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:50 smithi171 ceph-mon[32337]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:50 smithi186 ceph-mon[32753]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:51.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:50 smithi111 ceph-mon[27746]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:52.957 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:52 smithi171 ceph-mon[32337]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:52 smithi186 ceph-mon[32753]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:53.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:52 smithi111 ceph-mon[27746]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:54.983 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:54 smithi171 ceph-mon[32337]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:54.984 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T22:37:54.984 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:54 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:54 smithi186 ceph-mon[32753]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T22:37:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:54 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:55.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:54 smithi111 ceph-mon[27746]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T22:37:55.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:54 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:55.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:55 smithi186 ceph-mon[32753]: Deploying daemon osd.10 on smithi171 2024-04-24T22:37:56.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:55 smithi111 ceph-mon[27746]: Deploying daemon osd.10 on smithi171 2024-04-24T22:37:56.058 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:55 smithi171 ceph-mon[32337]: Deploying daemon osd.10 on smithi171 2024-04-24T22:37:56.993 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:56 smithi171 ceph-mon[32337]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:56 smithi186 ceph-mon[32753]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:57.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:56 smithi111 ceph-mon[27746]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:37:58 smithi186 ceph-mon[32753]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:59.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:37:58 smithi111 ceph-mon[27746]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:37:59.111 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:37:58 smithi171 ceph-mon[32337]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:00 smithi171 ceph-mon[32337]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:00 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:00 smithi186 ceph-mon[32753]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:00 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:00 smithi111 ceph-mon[27746]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:00 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:02.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:02 smithi171 ceph-mon[32337]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:02.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:02 smithi186 ceph-mon[32753]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:02 smithi111 ceph-mon[27746]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:03.195 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 10 on host 'smithi171' 2024-04-24T22:38:03.196 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.193+0000 7f3e097fa700 1 -- 172.21.15.171:0/1808429410 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3e0c0fe910 con 0x7f3df40a2ca0 2024-04-24T22:38:03.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 msgr2=0x7f3df40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:03.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 0x7f3df40a5160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b580 tx=0x7f3dfc005fb0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 msgr2=0x7f3e0c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400ef30 tx=0x7f3e0400c3a0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 shutdown_connections 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3df40a2ca0 0x7f3df40a5160 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e0c102d30 0x7f3e0c19fac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e0c101190 0x7f3e0c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 --2- 172.21.15.171:0/1808429410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c102390 0x7f3e0c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 >> 172.21.15.171:0/1808429410 conn(0x7f3e0c0fc750 msgr2=0x7f3e0c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 shutdown_connections 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-24T22:38:03.196+0000 7f3e1238c700 1 -- 172.21.15.171:0/1808429410 wait complete. 2024-04-24T22:38:03.770 DEBUG:teuthology.orchestra.run.smithi171:osd.10> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T22:38:03.773 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-04-24T22:38:03.773 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T22:38:04.023 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:38:04.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:04.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:05.496 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:05 smithi186 ceph-mon[32753]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T22:38:05.497 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:05 smithi186 ceph-mon[32753]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:05.497 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:05 smithi186 ceph-mon[32753]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T22:38:05.497 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:05 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:05.497 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:05 smithi186 ceph-mon[32753]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:05.515 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-mon[32337]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T22:38:05.515 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-mon[32337]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:05.515 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-mon[32337]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T22:38:05.515 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:05.515 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-mon[32337]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:05.515 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:38:05.180+0000 7f907b805700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T22:38:05.515 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:38:05 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:38:05.195+0000 7f9077df9700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:38:05.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:05 smithi111 ceph-mon[27746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T22:38:05.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:05 smithi111 ceph-mon[27746]: from='osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:05.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:05 smithi111 ceph-mon[27746]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T22:38:05.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:05 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:05.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:05 smithi111 ceph-mon[27746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114197 s, 918 MB/s 2024-04-24T22:38:06.085 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-24T22:38:06.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:06 smithi186 ceph-mon[32753]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:38:06.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:06 smithi186 ceph-mon[32753]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T22:38:06.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:06 smithi186 ceph-mon[32753]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:06.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:06 smithi111 ceph-mon[27746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:38:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:06 smithi111 ceph-mon[27746]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T22:38:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:06 smithi111 ceph-mon[27746]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:06 smithi171 ceph-mon[32337]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-24T22:38:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:06 smithi171 ceph-mon[32337]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T22:38:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:06 smithi171 ceph-mon[32337]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T22:38:06.717 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-04-24T22:38:06.960 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] boot 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:38:07.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:38:07.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] boot 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: osd.10 [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] boot 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:07.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:08.407 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi171 2024-04-24T22:38:08.407 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-24T22:38:08.407 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.407 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.407 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:08.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:08.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:08.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:08 smithi186 ceph-mon[32753]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi171 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:08.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:08 smithi111 ceph-mon[27746]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 -- 172.21.15.186:0/2327613783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0105640 msgr2=0x7f56c0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 --2- 172.21.15.186:0/2327613783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0105640 0x7f56c0107a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009990 tx=0x7f56a8009ca0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 -- 172.21.15.186:0/2327613783 shutdown_connections 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 --2- 172.21.15.186:0/2327613783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0107f70 0x7f56c010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 --2- 172.21.15.186:0/2327613783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56c0102d10 0x7f56c0105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 --2- 172.21.15.186:0/2327613783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0105640 0x7f56c0107a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 -- 172.21.15.186:0/2327613783 >> 172.21.15.186:0/2327613783 conn(0x7f56c00fc720 msgr2=0x7f56c00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:08.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 -- 172.21.15.186:0/2327613783 shutdown_connections 2024-04-24T22:38:08.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.618+0000 7f56c552a700 1 -- 172.21.15.186:0/2327613783 wait complete. 2024-04-24T22:38:08.621 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 Processor -- start 2024-04-24T22:38:08.622 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 -- start start 2024-04-24T22:38:08.622 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:08.622 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0105640 0x7f56c0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56c0107f70 0x7f56c019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010df50 con 0x7f56c0107f70 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010ddd0 con 0x7f56c0105640 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56c552a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010e0d0 con 0x7f56c0102d10 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58100/0 (socket says 172.21.15.186:58100) 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 -- 172.21.15.186:0/1691080847 learned_addr learned my addr 172.21.15.186:0/1691080847 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56b67fc700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0105640 0x7f56c0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 -- 172.21.15.186:0/1691080847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0105640 msgr2=0x7f56c0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0105640 0x7f56c0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 -- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56c0107f70 msgr2=0x7f56c019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56c0107f70 0x7f56c019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.619+0000 7f56beffd700 1 -- 172.21.15.186:0/1691080847 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c01a0230 con 0x7f56c0102d10 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56beffd700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b560 tx=0x7f56b000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:08.623 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0015020 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c01a0520 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b0011d10 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b000f2e0 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.620+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a0b30 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.621+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c0068f20 con 0x7f56c0102d10 2024-04-24T22:38:08.624 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.622+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f56b0018020 con 0x7f56c0102d10 2024-04-24T22:38:08.625 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.622+0000 7f56bcff9700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 0x7f56ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:08.625 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.623+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f56b00ce790 con 0x7f56c0102d10 2024-04-24T22:38:08.625 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.623+0000 7f56b67fc700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 0x7f56ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:08.625 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.624+0000 7f56b67fc700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 0x7f56ac0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b5c0 tx=0x7f56a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:08.627 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.626+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f56b001f020 con 0x7f56c0102d10 2024-04-24T22:38:08.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi171 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:08 smithi171 ceph-mon[32337]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T22:38:08.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:08.834+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f56c0062670 con 0x7f56ac0a2c20 2024-04-24T22:38:09.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:38:09.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:38:09.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:09.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:09 smithi186 ceph-mon[32753]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T22:38:09.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:38:09.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:38:09.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:09.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:09 smithi111 ceph-mon[27746]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T22:38:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:38:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:38:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:09 smithi171 ceph-mon[32337]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T22:38:10.547 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:10 smithi186 ceph-mon[32753]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:10.548 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:10 smithi186 ceph-mon[32753]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:10.548 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:10 smithi186 ceph-mon[32753]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T22:38:10.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:10 smithi111 ceph-mon[27746]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:10.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:10 smithi111 ceph-mon[27746]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:10.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:10 smithi111 ceph-mon[27746]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T22:38:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:10 smithi171 ceph-mon[32337]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:10 smithi171 ceph-mon[32337]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:10 smithi171 ceph-mon[32337]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T22:38:11.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/2303209436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:11 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:11 smithi186 ceph-mon[32753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]': finished 2024-04-24T22:38:11.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:11 smithi186 ceph-mon[32753]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T22:38:11.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:11 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/2303209436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:11 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:11 smithi111 ceph-mon[27746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]': finished 2024-04-24T22:38:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:11 smithi111 ceph-mon[27746]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T22:38:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:11 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:11.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/2303209436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:11 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]: dispatch 2024-04-24T22:38:11.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:11 smithi171 ceph-mon[32337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "add0e95d-86a9-4c00-8c5e-e775d9afabbf"}]': finished 2024-04-24T22:38:11.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:11 smithi171 ceph-mon[32337]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T22:38:11.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:11 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:12 smithi111 ceph-mon[27746]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.186:0/963220440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:38:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:12 smithi171 ceph-mon[32337]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:12.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.186:0/963220440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:38:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:12 smithi186 ceph-mon[32753]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.186:0/963220440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:38:14.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:14 smithi171 ceph-mon[32337]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:38:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:14 smithi186 ceph-mon[32753]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:38:15.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:14 smithi111 ceph-mon[27746]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-24T22:38:16.834 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:16 smithi186 ceph-mon[32753]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:38:16.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:16 smithi171 ceph-mon[32337]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:38:17.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:16 smithi111 ceph-mon[27746]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-24T22:38:18.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:18 smithi171 ceph-mon[32337]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:38:18.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:18 smithi186 ceph-mon[32753]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:38:19.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:18 smithi111 ceph-mon[27746]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T22:38:19.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T22:38:19.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:19.913 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T22:38:19.914 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:20.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T22:38:20.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:20.898 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:20 smithi186 ceph-mon[32753]: Deploying daemon osd.11 on smithi186 2024-04-24T22:38:20.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:20 smithi186 ceph-mon[32753]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:38:20.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:20 smithi171 ceph-mon[32337]: Deploying daemon osd.11 on smithi186 2024-04-24T22:38:20.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:20 smithi171 ceph-mon[32337]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:38:21.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:20 smithi111 ceph-mon[27746]: Deploying daemon osd.11 on smithi186 2024-04-24T22:38:21.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:20 smithi111 ceph-mon[27746]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T22:38:22.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:22 smithi186 ceph-mon[32753]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-24T22:38:22.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:22 smithi171 ceph-mon[32337]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-24T22:38:23.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:22 smithi111 ceph-mon[27746]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-24T22:38:24.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:24 smithi171 ceph-mon[32337]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T22:38:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:24.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:24 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:24 smithi186 ceph-mon[32753]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T22:38:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:25.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:25.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:24 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:25.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:24 smithi111 ceph-mon[27746]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T22:38:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:24 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:26.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:26 smithi171 ceph-mon[32337]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:26 smithi186 ceph-mon[32753]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:27.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:26 smithi111 ceph-mon[27746]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:28.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.377+0000 7f56bcff9700 1 -- 172.21.15.186:0/1691080847 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f56c0062670 con 0x7f56ac0a2c20 2024-04-24T22:38:28.379 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 msgr2=0x7f56ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 0x7f56ac0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b5c0 tx=0x7f56a8005fb0 comp rx=0 tx=0).stop 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 msgr2=0x7f56c01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b560 tx=0x7f56b000b920 comp rx=0 tx=0).stop 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 shutdown_connections 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56c0107f70 0x7f56c019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56c0102d10 0x7f56c01985d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f56ac0a2c20 0x7f56ac0a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:28.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 --2- 172.21.15.186:0/1691080847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0105640 0x7f56c0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:28.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 >> 172.21.15.186:0/1691080847 conn(0x7f56c00fc720 msgr2=0x7f56c01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:28.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 shutdown_connections 2024-04-24T22:38:28.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:28.380+0000 7f56c552a700 1 -- 172.21.15.186:0/1691080847 wait complete. 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:28 smithi171 ceph-mon[32337]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.673 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:28 smithi186 ceph-mon[32753]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:28.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:28.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:28 smithi111 ceph-mon[27746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T22:38:28.989 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T22:38:28.991 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T22:38:28.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T22:38:29.442 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:38:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T22:38:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T22:38:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.427 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:30 smithi186 ceph-mon[32753]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T22:38:30.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T22:38:30.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:30.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:30 smithi171 ceph-mon[32337]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: from='osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-24T22:38:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:30 smithi111 ceph-mon[27746]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:31.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-mon[32753]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:38:31.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-mon[32753]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T22:38:31.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-mon[32753]: from='osd.11 ' entity='osd.11' 2024-04-24T22:38:31.500 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:38:31.192+0000 7faacd9b5700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T22:38:31.500 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:38:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:38:31.214+0000 7faac5fa1700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:38:31.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:31 smithi111 ceph-mon[27746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:38:31.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:31 smithi111 ceph-mon[27746]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T22:38:31.532 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.532 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.532 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:31 smithi111 ceph-mon[27746]: from='osd.11 ' entity='osd.11' 2024-04-24T22:38:31.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:31 smithi171 ceph-mon[32337]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-24T22:38:31.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:31 smithi171 ceph-mon[32337]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T22:38:31.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:31.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:31 smithi171 ceph-mon[32337]: from='osd.11 ' entity='osd.11' 2024-04-24T22:38:31.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 -- 172.21.15.111:0/1057797465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 msgr2=0x7f1d48105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:31.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/1057797465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d48105110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009990 tx=0x7f1d3c009ca0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 -- 172.21.15.111:0/1057797465 shutdown_connections 2024-04-24T22:38:31.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/1057797465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48107f80 0x7f1d4810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/1057797465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d48105110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/1057797465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48105650 0x7f1d48107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 -- 172.21.15.111:0/1057797465 >> 172.21.15.111:0/1057797465 conn(0x7f1d480fc730 msgr2=0x7f1d480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 -- 172.21.15.111:0/1057797465 shutdown_connections 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.698+0000 7f1d4fc00700 1 -- 172.21.15.111:0/1057797465 wait complete. 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 Processor -- start 2024-04-24T22:38:31.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 -- start start 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d4819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48107f80 0x7f1d481a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810de70 con 0x7f1d48102d20 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810dcf0 con 0x7f1d48105650 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810dff0 con 0x7f1d48107f80 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.700+0000 7f1d4d200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d4819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:54504/0 (socket says 172.21.15.111:54504) 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d4d200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d4819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60516/0 (socket says 172.21.15.111:60516) 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 -- 172.21.15.111:0/148556760 learned_addr learned my addr 172.21.15.111:0/148556760 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:31.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d4dc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48107f80 0x7f1d481a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 -- 172.21.15.111:0/148556760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48107f80 msgr2=0x7f1d481a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48107f80 0x7f1d481a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 -- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 msgr2=0x7f1d4819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d4819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.701+0000 7f1d47e00700 1 -- 172.21.15.111:0/148556760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d481a2370 con 0x7f1d48105650 2024-04-24T22:38:31.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d47e00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400eb30 tx=0x7f1d3400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cbd0 con 0x7f1d48105650 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d34004500 con 0x7f1d48105650 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010430 con 0x7f1d48105650 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d481a2660 con 0x7f1d48105650 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.702+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d481a2b50 con 0x7f1d48105650 2024-04-24T22:38:31.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.703+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d48109b00 con 0x7f1d48105650 2024-04-24T22:38:31.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.707+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1d34043ae0 con 0x7f1d48105650 2024-04-24T22:38:31.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.708+0000 7f1d45600700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 0x7f1d300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:31.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.709+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f1d340ced80 con 0x7f1d48105650 2024-04-24T22:38:31.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.709+0000 7f1d4d200700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 0x7f1d300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:31.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.709+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d3409c200 con 0x7f1d48105650 2024-04-24T22:38:31.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.710+0000 7f1d4d200700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 0x7f1d300a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00b7b0 tx=0x7f1d3c00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:31.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.937+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1d4802dbf0 con 0x7f1d48105650 2024-04-24T22:38:31.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.938+0000 7f1d45600700 1 -- 172.21.15.111:0/148556760 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f1d3409bb00 con 0x7f1d48105650 2024-04-24T22:38:31.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 msgr2=0x7f1d300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 0x7f1d300a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00b7b0 tx=0x7f1d3c00b880 comp rx=0 tx=0).stop 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 msgr2=0x7f1d4819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400eb30 tx=0x7f1d3400ee40 comp rx=0 tx=0).stop 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 shutdown_connections 2024-04-24T22:38:31.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d48107f80 0x7f1d481a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d48102d20 0x7f1d4819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f1d300a2ca0 0x7f1d300a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 --2- 172.21.15.111:0/148556760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d48105650 0x7f1d4819ac50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 >> 172.21.15.111:0/148556760 conn(0x7f1d480fc730 msgr2=0x7f1d481025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 shutdown_connections 2024-04-24T22:38:31.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:31.941+0000 7f1d4fc00700 1 -- 172.21.15.111:0/148556760 wait complete. 2024-04-24T22:38:32.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: purged_snaps scrub starts 2024-04-24T22:38:32.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: purged_snaps scrub ok 2024-04-24T22:38:32.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/148556760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] boot 2024-04-24T22:38:32.654 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T22:38:32.654 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: purged_snaps scrub starts 2024-04-24T22:38:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: purged_snaps scrub ok 2024-04-24T22:38:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/148556760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] boot 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T22:38:32.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:32.694 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1713998286,"num_in_osds":12,"osd_in_since":1713998291,"num_remapped_pgs":0} 2024-04-24T22:38:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: purged_snaps scrub starts 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: purged_snaps scrub ok 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/148556760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: osd.11 [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] boot 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T22:38:32.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:38:33.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:33 smithi171 ceph-mon[32337]: Detected new or changed devices on smithi186 2024-04-24T22:38:33.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:33 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-24T22:38:33.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:33 smithi171 ceph-mon[32337]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T22:38:33.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T22:38:33.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:33 smithi186 ceph-mon[32753]: Detected new or changed devices on smithi186 2024-04-24T22:38:33.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:33 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-24T22:38:33.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:33 smithi186 ceph-mon[32753]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T22:38:33.774 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:33 smithi111 ceph-mon[27746]: Detected new or changed devices on smithi186 2024-04-24T22:38:33.775 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:33 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-24T22:38:33.775 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:33 smithi111 ceph-mon[27746]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T22:38:34.120 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:38:34.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:34 smithi111 ceph-mon[27746]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:34 smithi186 ceph-mon[32753]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:34.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:34 smithi171 ceph-mon[32337]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 -- 172.21.15.111:0/846169121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 msgr2=0x7fca901047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 --2- 172.21.15.111:0/846169121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca901047f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca840099c0 tx=0x7fca84009cd0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 -- 172.21.15.111:0/846169121 shutdown_connections 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 --2- 172.21.15.111:0/846169121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 0x7fca9010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 --2- 172.21.15.111:0/846169121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca901047f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 --2- 172.21.15.111:0/846169121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca90105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 -- 172.21.15.111:0/846169121 >> 172.21.15.111:0/846169121 conn(0x7fca900ff970 msgr2=0x7fca90101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:36.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 -- 172.21.15.111:0/846169121 shutdown_connections 2024-04-24T22:38:36.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.309+0000 7fca96e00700 1 -- 172.21.15.111:0/846169121 wait complete. 2024-04-24T22:38:36.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 Processor -- start 2024-04-24T22:38:36.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 -- start start 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca9019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 0x7fca901a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fca9010ffe0 con 0x7fca901043d0 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca9010fe60 con 0x7fca90105f70 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca96e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fca90110160 con 0x7fca901055d0 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca95000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca95a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca9019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca95000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34870/0 (socket says 172.21.15.111:34870) 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.311+0000 7fca95000700 1 -- 172.21.15.111:0/979048202 learned_addr learned my addr 172.21.15.111:0/979048202 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca96400700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 0x7fca901a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 -- 172.21.15.111:0/979048202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 msgr2=0x7fca901a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:36.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 0x7fca901a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 -- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 msgr2=0x7fca9019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca9019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 -- 172.21.15.111:0/979048202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901a2280 con 0x7fca901055d0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca95000700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00eb60 tx=0x7fca7c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00cbd0 con 0x7fca901055d0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7c004500 con 0x7fca901055d0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a2570 con 0x7fca901055d0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.312+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c010430 con 0x7fca901055d0 2024-04-24T22:38:36.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.313+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a2b00 con 0x7fca901055d0 2024-04-24T22:38:36.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.314+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca90068f20 con 0x7fca901055d0 2024-04-24T22:38:36.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.315+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fca7c003680 con 0x7fca901055d0 2024-04-24T22:38:36.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.317+0000 7fca8e000700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 0x7fca780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:36.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.317+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fca7c0ceda0 con 0x7fca901055d0 2024-04-24T22:38:36.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.317+0000 7fca95a00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 0x7fca780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:36.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.318+0000 7fca95a00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 0x7fca780a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b580 tx=0x7fca84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:36.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.321+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fca7c00cd70 con 0x7fca901055d0 2024-04-24T22:38:36.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.545+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fca90062e40 con 0x7fca901055d0 2024-04-24T22:38:36.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.546+0000 7fca8e000700 1 -- 172.21.15.111:0/979048202 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fca7c09c100 con 0x7fca901055d0 2024-04-24T22:38:36.547 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 msgr2=0x7fca780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 0x7fca780a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b580 tx=0x7fca84005fb0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 msgr2=0x7fca9019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00eb60 tx=0x7fca7c00ee70 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 shutdown_connections 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fca780a2ca0 0x7fca780a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca90105f70 0x7fca901a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca901043d0 0x7fca9019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 --2- 172.21.15.111:0/979048202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca901055d0 0x7fca9019ab60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 >> 172.21.15.111:0/979048202 conn(0x7fca900ff970 msgr2=0x7fca901089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:36.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 shutdown_connections 2024-04-24T22:38:36.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:36.549+0000 7fca96e00700 1 -- 172.21.15.111:0/979048202 wait complete. 2024-04-24T22:38:36.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:36 smithi186 ceph-mon[32753]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:36.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:36 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/979048202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:36.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:36 smithi171 ceph-mon[32337]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:36.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:36 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/979048202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:36.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:36 smithi111 ceph-mon[27746]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:36.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:36 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/979048202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:38:37.262 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713998312,"num_in_osds":12,"osd_in_since":1713998291,"num_remapped_pgs":0} 2024-04-24T22:38:37.262 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:38:37.706 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:38:38.870 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:38 smithi111 ceph-mon[27746]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:38.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:38 smithi171 ceph-mon[32337]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:38.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:38 smithi186 ceph-mon[32753]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:39.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.684+0000 7f2062400700 1 -- 172.21.15.111:0/650199440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 msgr2=0x7f205c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:39.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.684+0000 7f2062400700 1 --2- 172.21.15.111:0/650199440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c106e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2048009990 tx=0x7f2048009ca0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 -- 172.21.15.111:0/650199440 shutdown_connections 2024-04-24T22:38:39.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 --2- 172.21.15.111:0/650199440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c107570 0x7f205c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 --2- 172.21.15.111:0/650199440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c1024e0 0x7f205c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 --2- 172.21.15.111:0/650199440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c106e90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 -- 172.21.15.111:0/650199440 >> 172.21.15.111:0/650199440 conn(0x7f205c079390 msgr2=0x7f205c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 -- 172.21.15.111:0/650199440 shutdown_connections 2024-04-24T22:38:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.685+0000 7f2062400700 1 -- 172.21.15.111:0/650199440 wait complete. 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.686+0000 7f2062400700 1 Processor -- start 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.686+0000 7f2062400700 1 -- start start 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 0x7f205c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f205c112580 con 0x7f205c107570 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f205c112400 con 0x7f205c1024e0 2024-04-24T22:38:39.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f2062400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f205c112700 con 0x7f205c102e90 2024-04-24T22:38:39.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f205b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:39.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f205b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:46978/0 (socket says 172.21.15.111:46978) 2024-04-24T22:38:39.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f205b400700 1 -- 172.21.15.111:0/3953559344 learned_addr learned my addr 172.21.15.111:0/3953559344 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:39.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f205be00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 0x7f205c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:39.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.687+0000 7f205aa00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 -- 172.21.15.111:0/3953559344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 msgr2=0x7f205c074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 -- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 msgr2=0x7f205c074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 0x7f205c074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 -- 172.21.15.111:0/3953559344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1a6fa0 con 0x7f205c1024e0 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205be00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 0x7f205c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f205b400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f204c00e9a0 tx=0x7f204c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c00ca10 con 0x7f205c1024e0 2024-04-24T22:38:39.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204c00cbb0 con 0x7f205c1024e0 2024-04-24T22:38:39.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a7290 con 0x7f205c1024e0 2024-04-24T22:38:39.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.688+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c018680 con 0x7f205c1024e0 2024-04-24T22:38:39.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.689+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a77a0 con 0x7f205c1024e0 2024-04-24T22:38:39.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.689+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c103970 con 0x7f205c1024e0 2024-04-24T22:38:39.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.692+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f204c010430 con 0x7f205c1024e0 2024-04-24T22:38:39.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.693+0000 7f2053e00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 0x7f20400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:39.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.693+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f204c0cee60 con 0x7f205c1024e0 2024-04-24T22:38:39.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.693+0000 7f205aa00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 0x7f20400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:39.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.694+0000 7f205aa00700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 0x7f20400a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2048006010 tx=0x7f2048005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:39.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.697+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f204c09c1c0 con 0x7f205c1024e0 2024-04-24T22:38:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.923+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f205c02db30 con 0x7f205c1024e0 2024-04-24T22:38:39.926 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.925+0000 7f2053e00700 1 -- 172.21.15.111:0/3953559344 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7f204c09bac0 con 0x7f205c1024e0 2024-04-24T22:38:39.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:39.928 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":81,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:38:33.199022+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 msgr2=0x7f20400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 0x7f20400a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2048006010 tx=0x7f2048005a90 comp rx=0 tx=0).stop 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 msgr2=0x7f205c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f204c00e9a0 tx=0x7f204c00ecb0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 shutdown_connections 2024-04-24T22:38:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c107570 0x7f205c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f20400a2c20 0x7f20400a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f205c1024e0 0x7f205c074530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 --2- 172.21.15.111:0/3953559344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f205c102e90 0x7f205c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.927+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 >> 172.21.15.111:0/3953559344 conn(0x7f205c079390 msgr2=0x7f205c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.928+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 shutdown_connections 2024-04-24T22:38:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:39.928+0000 7f2062400700 1 -- 172.21.15.111:0/3953559344 wait complete. 2024-04-24T22:38:40.729 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:38:40.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T22:38:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:40 smithi171 ceph-mon[32337]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:40 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3953559344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:40 smithi186 ceph-mon[32753]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:40 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3953559344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:41.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:40 smithi111 ceph-mon[27746]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:41.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:40 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3953559344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:38:42.890 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:42 smithi111 ceph-mon[27746]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:42.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:42 smithi171 ceph-mon[32337]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:42 smithi186 ceph-mon[32753]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 -- 172.21.15.111:0/3123459583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 msgr2=0x7f82dc10f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 --2- 172.21.15.111:0/3123459583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc10f140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82d400b0a0 tx=0x7f82d400b3b0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 -- 172.21.15.111:0/3123459583 shutdown_connections 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 --2- 172.21.15.111:0/3123459583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc10f140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 --2- 172.21.15.111:0/3123459583 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc1024e0 0x7f82dc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.166+0000 7f82e2a00700 1 --2- 172.21.15.111:0/3123459583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc102e90 0x7f82dc106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.167+0000 7f82e2a00700 1 -- 172.21.15.111:0/3123459583 >> 172.21.15.111:0/3123459583 conn(0x7f82dc079390 msgr2=0x7f82dc0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.167+0000 7f82e2a00700 1 -- 172.21.15.111:0/3123459583 shutdown_connections 2024-04-24T22:38:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.167+0000 7f82e2a00700 1 -- 172.21.15.111:0/3123459583 wait complete. 2024-04-24T22:38:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 Processor -- start 2024-04-24T22:38:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 -- start start 2024-04-24T22:38:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 0x7f82dc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 0x7f82dc1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc112550 con 0x7f82dc1024e0 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1123d0 con 0x7f82dc102e90 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.168+0000 7f82e2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1126d0 con 0x7f82dc107570 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 0x7f82dc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82e0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82db400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 0x7f82dc1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82e0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34888/0 (socket says 172.21.15.111:34888) 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82e0a00700 1 -- 172.21.15.111:0/2602680419 learned_addr learned my addr 172.21.15.111:0/2602680419 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 -- 172.21.15.111:0/2602680419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 msgr2=0x7f82dc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 -- 172.21.15.111:0/2602680419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 msgr2=0x7f82dc1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 0x7f82dc1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82dbe00700 1 -- 172.21.15.111:0/2602680419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19b100 con 0x7f82dc1024e0 2024-04-24T22:38:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.169+0000 7f82db400700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 0x7f82dc1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82dbe00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 0x7f82dc1a07e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00e9f0 tx=0x7f82cc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc004220 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc004d10 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc19b3f0 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005020 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.170+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a76b0 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.171+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc068f20 con 0x7f82dc1024e0 2024-04-24T22:38:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.173+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82cc004750 con 0x7f82dc1024e0 2024-04-24T22:38:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.175+0000 7f82d8c00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 0x7f82c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.175+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f82cc0cebd0 con 0x7f82dc1024e0 2024-04-24T22:38:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.175+0000 7f82db400700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 0x7f82c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.176+0000 7f82db400700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 0x7f82c00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82dc10f4f0 tx=0x7f82c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.178+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82cc09bfc0 con 0x7f82dc1024e0 2024-04-24T22:38:43.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.396+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82dc1a78e0 con 0x7f82dc1024e0 2024-04-24T22:38:43.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.397+0000 7f82d8c00700 1 -- 172.21.15.111:0/2602680419 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f82cc09b8c0 con 0x7f82dc1024e0 2024-04-24T22:38:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:38:43.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 msgr2=0x7f82c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:43.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 0x7f82c00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82dc10f4f0 tx=0x7f82c8009260 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 msgr2=0x7f82dc1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 0x7f82dc1a07e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00e9f0 tx=0x7f82cc00ed00 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 shutdown_connections 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82dc107570 0x7f82dc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82dc1024e0 0x7f82dc1a07e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82c00a2c20 0x7f82c00a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 --2- 172.21.15.111:0/2602680419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82dc102e90 0x7f82dc1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.400+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 >> 172.21.15.111:0/2602680419 conn(0x7f82dc079390 msgr2=0x7f82dc1003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.401+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 shutdown_connections 2024-04-24T22:38:43.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:43.401+0000 7f82e2a00700 1 -- 172.21.15.111:0/2602680419 wait complete. 2024-04-24T22:38:43.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2602680419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:43.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2602680419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2602680419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:44.169 INFO:tasks.cephadm:Adding mds.a on smithi111 2024-04-24T22:38:44.169 INFO:tasks.cephadm:Adding mds.d on smithi111 2024-04-24T22:38:44.169 INFO:tasks.cephadm:Adding mds.g on smithi111 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.j on smithi111 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.b on smithi171 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.e on smithi171 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.h on smithi171 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.k on smithi171 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-04-24T22:38:44.170 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-04-24T22:38:44.170 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-04-24T22:38:44.421 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:38:44.856 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:44 smithi186 ceph-mon[32753]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:44.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:44 smithi171 ceph-mon[32337]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:45.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:44 smithi111 ceph-mon[27746]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:45.993 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- 172.21.15.186:0/2306630944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101190 msgr2=0x7f77c81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:45.993 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 --2- 172.21.15.186:0/2306630944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101190 0x7f77c81015b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009990 tx=0x7f77b8009ca0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.993 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- 172.21.15.186:0/2306630944 shutdown_connections 2024-04-24T22:38:45.993 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 --2- 172.21.15.186:0/2306630944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c8102d30 0x7f77c81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 --2- 172.21.15.186:0/2306630944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101190 0x7f77c81015b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 --2- 172.21.15.186:0/2306630944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8102390 0x7f77c81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- 172.21.15.186:0/2306630944 >> 172.21.15.186:0/2306630944 conn(0x7f77c80fc750 msgr2=0x7f77c80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- 172.21.15.186:0/2306630944 shutdown_connections 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- 172.21.15.186:0/2306630944 wait complete. 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 Processor -- start 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.990+0000 7f77cd423700 1 -- start start 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8101190 0x7f77c8197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c8102390 0x7f77c81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77c810ce30 con 0x7f77c8102390 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77c810ccb0 con 0x7f77c8101190 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77c810cfb0 con 0x7f77c8102d30 2024-04-24T22:38:45.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:56874/0 (socket says 172.21.15.186:56874) 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 -- 172.21.15.186:0/2249873822 learned_addr learned my addr 172.21.15.186:0/2249873822 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c6ffd700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8101190 0x7f77c8197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 -- 172.21.15.186:0/2249873822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8101190 msgr2=0x7f77c8197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8101190 0x7f77c8197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 -- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c8102390 msgr2=0x7f77c81979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c8102390 0x7f77c81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 -- 172.21.15.186:0/2249873822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c819f0d0 con 0x7f77c8102d30 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77c77fe700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77c000b9a0 tx=0x7f77c000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0019070 con 0x7f77c8102d30 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c000ce10 con 0x7f77c8102d30 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c819f3c0 con 0x7f77c8102d30 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c00073d0 con 0x7f77c8102d30 2024-04-24T22:38:45.995 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.991+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c806ac20 con 0x7f77c8102d30 2024-04-24T22:38:45.996 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.993+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c806b710 con 0x7f77c8102d30 2024-04-24T22:38:45.996 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.993+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f77c001c070 con 0x7f77c8102d30 2024-04-24T22:38:45.996 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.994+0000 7f77affff700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 0x7f77b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:45.996 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.994+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f77c00cdf20 con 0x7f77c8102d30 2024-04-24T22:38:45.996 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.994+0000 7f77c6ffd700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 0x7f77b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:45.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.995+0000 7f77c6ffd700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 0x7f77b00a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:46.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:45.998+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77c009b320 con 0x7f77c8102d30 2024-04-24T22:38:46.204 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.203+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) -- 0x7f77c8108400 con 0x7f77b00a2c20 2024-04-24T22:38:46.221 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.220+0000 7f77affff700 1 -- 172.21.15.186:0/2249873822 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f77c8108400 con 0x7f77b00a2c20 2024-04-24T22:38:46.222 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 msgr2=0x7f77b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 0x7f77b00a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8005fb0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 msgr2=0x7f77c819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77c000b9a0 tx=0x7f77c000bcb0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 shutdown_connections 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102d30 0x7f77c819ea30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f77b00a2c20 0x7f77b00a50e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c8101190 0x7f77c8197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 --2- 172.21.15.186:0/2249873822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c8102390 0x7f77c81979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:46.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 >> 172.21.15.186:0/2249873822 conn(0x7f77c80fc750 msgr2=0x7f77c8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:46.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 shutdown_connections 2024-04-24T22:38:46.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-24T22:38:46.222+0000 7f77cd423700 1 -- 172.21.15.186:0/2249873822 wait complete. 2024-04-24T22:38:46.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:46 smithi171 ceph-mon[32337]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:46.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:46 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:46.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:46 smithi186 ceph-mon[32753]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:46.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:46.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:46 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:46.960 DEBUG:teuthology.orchestra.run.smithi111:mds.a> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T22:38:46.966 DEBUG:teuthology.orchestra.run.smithi111:mds.d> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T22:38:47.012 DEBUG:teuthology.orchestra.run.smithi111:mds.g> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T22:38:47.015 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:46 smithi111 ceph-mon[27746]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:47.015 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:47.015 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:46 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:47.020 DEBUG:teuthology.orchestra.run.smithi111:mds.j> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T22:38:47.067 DEBUG:teuthology.orchestra.run.smithi171:mds.b> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T22:38:47.070 DEBUG:teuthology.orchestra.run.smithi171:mds.e> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T22:38:47.115 DEBUG:teuthology.orchestra.run.smithi171:mds.h> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T22:38:47.157 DEBUG:teuthology.orchestra.run.smithi171:mds.k> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T22:38:47.200 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T22:38:47.203 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T22:38:47.245 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T22:38:47.288 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T22:38:47.331 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T22:38:47.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 -- 172.21.15.111:0/947215103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc109620 msgr2=0x7ff5cc10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 --2- 172.21.15.111:0/947215103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc109620 0x7ff5cc10bac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009ca0 tx=0x7ff5c402a950 comp rx=0 tx=0).stop 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 -- 172.21.15.111:0/947215103 shutdown_connections 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 --2- 172.21.15.111:0/947215103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc109620 0x7ff5cc10bac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 --2- 172.21.15.111:0/947215103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc074140 0x7ff5cc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 --2- 172.21.15.111:0/947215103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.548+0000 7ff5d1800640 1 -- 172.21.15.111:0/947215103 >> 172.21.15.111:0/947215103 conn(0x7ff5cc0fcd20 msgr2=0x7ff5cc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.549+0000 7ff5d1800640 1 -- 172.21.15.111:0/947215103 shutdown_connections 2024-04-24T22:38:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.549+0000 7ff5d1800640 1 -- 172.21.15.111:0/947215103 wait complete. 2024-04-24T22:38:47.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 Processor -- start 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 -- start start 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 0x7ff5cc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc109620 0x7ff5cc1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10f660 con 0x7ff5cc075720 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10f4e0 con 0x7ff5cc109620 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5d1800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10f7e0 con 0x7ff5cc074140 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5caa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 0x7ff5cc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5caa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 0x7ff5cc199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:39058/0 (socket says 172.21.15.111:39058) 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5caa00640 1 -- 172.21.15.111:0/3867788410 learned_addr learned my addr 172.21.15.111:0/3867788410 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:47.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5ca000640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.550+0000 7ff5cb400640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc109620 0x7ff5cc1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 -- 172.21.15.111:0/3867788410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 msgr2=0x7ff5cc199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 0x7ff5cc199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 -- 172.21.15.111:0/3867788410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc109620 msgr2=0x7ff5cc1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc109620 0x7ff5cc1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 -- 172.21.15.111:0/3867788410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc1a1890 con 0x7ff5cc075720 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5ca000640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc19a1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e9f0 tx=0x7ff5b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800ccc0 con 0x7ff5cc075720 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8005020 con 0x7ff5cc075720 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc1a1b80 con 0x7ff5cc075720 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.551+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80049e0 con 0x7ff5cc075720 2024-04-24T22:38:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.552+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a2080 con 0x7ff5cc075720 2024-04-24T22:38:47.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.553+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5cc1083e0 con 0x7ff5cc075720 2024-04-24T22:38:47.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.556+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff5b8005560 con 0x7ff5cc075720 2024-04-24T22:38:47.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.557+0000 7ff5c3400640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 0x7ff5b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:47.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.558+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5b80cec20 con 0x7ff5cc075720 2024-04-24T22:38:47.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.558+0000 7ff5caa00640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 0x7ff5b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:47.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.558+0000 7ff5caa00640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 0x7ff5b00a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc19b150 tx=0x7ff5bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:47.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.562+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff5b80051d0 con 0x7ff5cc075720 2024-04-24T22:38:47.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.766+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5cc062440 con 0x7ff5cc075720 2024-04-24T22:38:47.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.768+0000 7ff5c3400640 1 -- 172.21.15.111:0/3867788410 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7ff5b809bf80 con 0x7ff5cc075720 2024-04-24T22:38:47.770 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:47.770 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":81,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:38:33.199022+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 msgr2=0x7ff5b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 0x7ff5b00a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc19b150 tx=0x7ff5bc005f90 comp rx=0 tx=0).stop 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 msgr2=0x7ff5cc19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc19a1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e9f0 tx=0x7ff5b800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 shutdown_connections 2024-04-24T22:38:47.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5b00a2b90 0x7ff5b00a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5cc109620 0x7ff5cc1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc075720 0x7ff5cc19a1c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 --2- 172.21.15.111:0/3867788410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc074140 0x7ff5cc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.770+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 >> 172.21.15.111:0/3867788410 conn(0x7ff5cc0fcd20 msgr2=0x7ff5cc1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.771+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 shutdown_connections 2024-04-24T22:38:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.771+0000 7ff5d1800640 1 -- 172.21.15.111:0/3867788410 wait complete. 2024-04-24T22:38:47.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:47.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:47 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:47.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:38:47.793 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:38:47.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:47.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:47 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-24T22:38:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:48.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:47 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 -- 172.21.15.111:0/1910135702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 msgr2=0x7f42f8106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 --2- 172.21.15.111:0/1910135702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8106df0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 -- 172.21.15.111:0/1910135702 shutdown_connections 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 --2- 172.21.15.111:0/1910135702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f8107330 0x7f42f8109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 --2- 172.21.15.111:0/1910135702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8106df0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 --2- 172.21.15.111:0/1910135702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f81020d0 0x7f42f81044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 -- 172.21.15.111:0/1910135702 >> 172.21.15.111:0/1910135702 conn(0x7f42f80fbae0 msgr2=0x7f42f80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 -- 172.21.15.111:0/1910135702 shutdown_connections 2024-04-24T22:38:48.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:47.999+0000 7f42fca00640 1 -- 172.21.15.111:0/1910135702 wait complete. 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 Processor -- start 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 -- start start 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f81020d0 0x7f42f8101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 0x7f42f81005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810d2b0 con 0x7f42f81020d0 2024-04-24T22:38:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810d130 con 0x7f42f8107330 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.000+0000 7f42fca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810d430 con 0x7f42f8104a00 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42f7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f81020d0 0x7f42f8101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:39082/0 (socket says 172.21.15.111:39082) 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 -- 172.21.15.111:0/26444207 learned_addr learned my addr 172.21.15.111:0/26444207 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42f7e00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 0x7f42f81005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 -- 172.21.15.111:0/26444207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 msgr2=0x7f42f81005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 0x7f42f81005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 -- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f81020d0 msgr2=0x7f42f8101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f81020d0 0x7f42f8101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 -- 172.21.15.111:0/26444207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8100e50 con 0x7f42f8104a00 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42f7e00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 0x7f42f81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:38:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42eea00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42e0031610 tx=0x7f42e0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e003f030 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0003ad0 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.001+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f81a1a20 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.002+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0035070 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.002+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a1f70 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.003+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f80fd5d0 con 0x7f42f8104a00 2024-04-24T22:38:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.004+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f42e0042030 con 0x7f42f8104a00 2024-04-24T22:38:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.006+0000 7f42f4c00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 0x7f42dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.006+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f42e00ef280 con 0x7f42f8104a00 2024-04-24T22:38:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.006+0000 7f42f7400640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 0x7f42dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.007+0000 7f42f7400640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 0x7f42dc0a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42e80095f0 tx=0x7f42e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.010+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42e00bc5e0 con 0x7f42f8104a00 2024-04-24T22:38:48.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.210+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42f8062340 con 0x7f42f8104a00 2024-04-24T22:38:48.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.211+0000 7f42f4c00640 1 -- 172.21.15.111:0/26444207 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f42e00bbee0 con 0x7f42f8104a00 2024-04-24T22:38:48.213 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:38:48.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.213+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 msgr2=0x7f42dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.213+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 0x7f42dc0a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42e80095f0 tx=0x7f42e8005f90 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.213+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 msgr2=0x7f42f8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.213+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42e0031610 tx=0x7f42e0004270 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 shutdown_connections 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f42dc0a2b90 0x7f42dc0a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42f8107330 0x7f42f81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42f8104a00 0x7f42f8100080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 --2- 172.21.15.111:0/26444207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42f81020d0 0x7f42f8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 >> 172.21.15.111:0/26444207 conn(0x7f42f80fbae0 msgr2=0x7f42f8102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:48.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 shutdown_connections 2024-04-24T22:38:48.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.214+0000 7f42fca00640 1 -- 172.21.15.111:0/26444207 wait complete. 2024-04-24T22:38:48.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:38:48.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 -- 172.21.15.111:0/3745692929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601049b0 msgr2=0x7f6460104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 --2- 172.21.15.111:0/3745692929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601049b0 0x7f6460104e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6454009900 tx=0x7f645402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 -- 172.21.15.111:0/3745692929 shutdown_connections 2024-04-24T22:38:48.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 --2- 172.21.15.111:0/3745692929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f646010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 --2- 172.21.15.111:0/3745692929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601049b0 0x7f6460104e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 --2- 172.21.15.111:0/3745692929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601037c0 0x7f6460103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 -- 172.21.15.111:0/3745692929 >> 172.21.15.111:0/3745692929 conn(0x7f64600fed50 msgr2=0x7f6460101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:48.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 -- 172.21.15.111:0/3745692929 shutdown_connections 2024-04-24T22:38:48.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.439+0000 7f6466600640 1 -- 172.21.15.111:0/3745692929 wait complete. 2024-04-24T22:38:48.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.440+0000 7f6466600640 1 Processor -- start 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 -- start start 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 0x7f6460199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601049b0 0x7f646019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f64601a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f646010f440 con 0x7f6460105370 2024-04-24T22:38:48.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f646010f2c0 con 0x7f64601049b0 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6466600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f646010f5c0 con 0x7f64601037c0 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6465c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f64601a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6465c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f64601a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48078/0 (socket says 172.21.15.111:48078) 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6465c00640 1 -- 172.21.15.111:0/2467553576 learned_addr learned my addr 172.21.15.111:0/2467553576 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f6465200640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 0x7f6460199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.441+0000 7f645fe00640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601049b0 0x7f646019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 -- 172.21.15.111:0/2467553576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601049b0 msgr2=0x7f646019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601049b0 0x7f646019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 -- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 msgr2=0x7f64601a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f64601a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 -- 172.21.15.111:0/2467553576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64601a16f0 con 0x7f64601037c0 2024-04-24T22:38:48.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6465200640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 0x7f6460199b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c00e9c0 tx=0x7f644c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00ccc0 con 0x7f64601037c0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c0035f0 con 0x7f64601037c0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64601a19e0 con 0x7f64601037c0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00ce60 con 0x7f64601037c0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.442+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64601a1f00 con 0x7f64601037c0 2024-04-24T22:38:48.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.444+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646010aae0 con 0x7f64601037c0 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.446+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f644c0037a0 con 0x7f64601037c0 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.448+0000 7f645d600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 0x7f64480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.448+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f644c0cf0e0 con 0x7f64601037c0 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.448+0000 7f645fe00640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 0x7f64480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.449+0000 7f645fe00640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 0x7f64480a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f64540314e0 tx=0x7f6454000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:48.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.451+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f644c09c440 con 0x7f64601037c0 2024-04-24T22:38:48.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.691+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f646002d170 con 0x7f64601037c0 2024-04-24T22:38:48.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.692+0000 7f645d600640 1 -- 172.21.15.111:0/2467553576 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f644c09bd40 con 0x7f64601037c0 2024-04-24T22:38:48.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:48.694 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T22:38:48.695 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 1 2024-04-24T22:38:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 msgr2=0x7f64480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 0x7f64480a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f64540314e0 tx=0x7f6454000bc0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 msgr2=0x7f6460199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 0x7f6460199b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c00e9c0 tx=0x7f644c00ee80 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 shutdown_connections 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64480a2c10 0x7f64480a50d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6460105370 0x7f64601a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64601049b0 0x7f646019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 --2- 172.21.15.111:0/2467553576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64601037c0 0x7f6460199b70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 >> 172.21.15.111:0/2467553576 conn(0x7f64600fed50 msgr2=0x7f6460107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 shutdown_connections 2024-04-24T22:38:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:48.695+0000 7f6466600640 1 -- 172.21.15.111:0/2467553576 wait complete. 2024-04-24T22:38:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:48 smithi186 ceph-mon[32753]: Deploying daemon mds.l on smithi186 2024-04-24T22:38:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:48 smithi186 ceph-mon[32753]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:48 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3867788410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:48 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/26444207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:48.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:48 smithi171 ceph-mon[32337]: Deploying daemon mds.l on smithi186 2024-04-24T22:38:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:48 smithi171 ceph-mon[32337]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:48 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3867788410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:48 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/26444207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:48 smithi111 ceph-mon[27746]: Deploying daemon mds.l on smithi186 2024-04-24T22:38:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:48 smithi111 ceph-mon[27746]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:48 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3867788410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:38:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:48 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/26444207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:38:49.643 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:49 smithi186 systemd[1]: Starting Ceph mds.l for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:38:49.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:49 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2467553576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:49.894 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:49 smithi186 podman[63083]: 2024-04-24 22:38:49.632976851 +0000 UTC m=+0.018298114 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:38:49.894 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:49 smithi186 podman[63083]: 2024-04-24 22:38:49.757389341 +0000 UTC m=+0.142710599 container create 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:38:49.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:49 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2467553576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:50.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:49 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2467553576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:50.249 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:49 smithi186 podman[63083]: 2024-04-24 22:38:49.976350959 +0000 UTC m=+0.361672220 container init 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T22:38:50.250 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:49 smithi186 podman[63083]: 2024-04-24 22:38:49.979084212 +0000 UTC m=+0.364405477 container start 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T22:38:50.250 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mds[63102]: -- 172.21.15.186:0/2725358211 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da24375a40 con 0x55da24348c00 2024-04-24T22:38:50.250 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l[63098]: starting mds.l at 2024-04-24T22:38:50.250 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:50 smithi186 bash[63083]: 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 2024-04-24T22:38:50.250 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:38:50 smithi186 systemd[1]: Started Ceph mds.l for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:38:50.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:38:50.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 -- 172.21.15.111:0/3850201097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0077340 msgr2=0x7fb9b00777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 --2- 172.21.15.111:0/3850201097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0077340 0x7fb9b00777a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009900 tx=0x7fb99c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 -- 172.21.15.111:0/3850201097 shutdown_connections 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 --2- 172.21.15.111:0/3850201097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077ce0 0x7fb9b0078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 --2- 172.21.15.111:0/3850201097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0077340 0x7fb9b00777a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 --2- 172.21.15.111:0/3850201097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0076100 0x7fb9b0076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.928+0000 7fb9afe00640 1 -- 172.21.15.111:0/3850201097 >> 172.21.15.111:0/3850201097 conn(0x7fb9b00fedb0 msgr2=0x7fb9b01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.929+0000 7fb9afe00640 1 -- 172.21.15.111:0/3850201097 shutdown_connections 2024-04-24T22:38:50.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.929+0000 7fb9afe00640 1 -- 172.21.15.111:0/3850201097 wait complete. 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 Processor -- start 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 -- start start 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 0x7fb9b019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077340 0x7fb9b019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:50.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01138a0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113720 con 0x7fb9b0076100 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9afe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113a20 con 0x7fb9b0077340 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9af400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9af400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48100/0 (socket says 172.21.15.111:48100) 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9af400640 1 -- 172.21.15.111:0/2844707647 learned_addr learned my addr 172.21.15.111:0/2844707647 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9aea00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 0x7fb9b019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.930+0000 7fb9ae000640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077340 0x7fb9b019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 -- 172.21.15.111:0/2844707647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077340 msgr2=0x7fb9b019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077340 0x7fb9b019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 -- 172.21.15.111:0/2844707647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 msgr2=0x7fb9b019dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 0x7fb9b019dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 -- 172.21.15.111:0/2844707647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b01a5ae0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9aea00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 0x7fb9b019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:38:50.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9af400640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800eb10 tx=0x7fb9a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004b20 con 0x7fb9b0077ce0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8004cc0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a5dd0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.931+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8010760 con 0x7fb9b0077ce0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.932+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a6290 con 0x7fb9b0077ce0 2024-04-24T22:38:50.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.933+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b01101b0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.935+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb9a8005020 con 0x7fb9b0077ce0 2024-04-24T22:38:50.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.936+0000 7fb9a7400640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 0x7fb9940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:50.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.937+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb9a80cf2e0 con 0x7fb9b0077ce0 2024-04-24T22:38:50.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.937+0000 7fb9aea00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 0x7fb9940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:50.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.937+0000 7fb9aea00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 0x7fb9940a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb9a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:50.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:50.941+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb9a809c640 con 0x7fb9b0077ce0 2024-04-24T22:38:50.963 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:50.963 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6832/1686750740,v1:172.21.15.186:6833/1686750740] up:boot 2024-04-24T22:38:50.963 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: fsmap 1 up:standby 2024-04-24T22:38:50.963 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:50.964 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:50 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6832/1686750740,v1:172.21.15.186:6833/1686750740] up:boot 2024-04-24T22:38:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: fsmap 1 up:standby 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:51.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:50 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6832/1686750740,v1:172.21.15.186:6833/1686750740] up:boot 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: fsmap 1 up:standby 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:51.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:51.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:51.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:50 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.180+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b00623e0 con 0x7fb9b0077ce0 2024-04-24T22:38:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.181+0000 7fb9a7400640 1 -- 172.21.15.111:0/2844707647 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb9a809bf40 con 0x7fb9b0077ce0 2024-04-24T22:38:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:51.183 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T22:38:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 2 2024-04-24T22:38:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 msgr2=0x7fb9940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 0x7fb9940a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb9a0005f90 comp rx=0 tx=0).stop 2024-04-24T22:38:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 msgr2=0x7fb9b01a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800eb10 tx=0x7fb9a800c2d0 comp rx=0 tx=0).stop 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 shutdown_connections 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9940a2b90 0x7fb9940a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b0077ce0 0x7fb9b01a5470 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0077340 0x7fb9b019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 --2- 172.21.15.111:0/2844707647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0076100 0x7fb9b019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.184+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 >> 172.21.15.111:0/2844707647 conn(0x7fb9b00fedb0 msgr2=0x7fb9b01008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.185+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 shutdown_connections 2024-04-24T22:38:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:51.185+0000 7fb9afe00640 1 -- 172.21.15.111:0/2844707647 wait complete. 2024-04-24T22:38:51.970 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:51 smithi171 ceph-mon[32337]: Deploying daemon mds.k on smithi171 2024-04-24T22:38:51.971 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:51 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2844707647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:51 smithi186 ceph-mon[32753]: Deploying daemon mds.k on smithi171 2024-04-24T22:38:52.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:51 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2844707647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:52.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:51 smithi111 ceph-mon[27746]: Deploying daemon mds.k on smithi171 2024-04-24T22:38:52.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:51 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2844707647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:52.658 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 podman[61201]: 2024-04-24 22:38:52.46449702 +0000 UTC m=+0.018280580 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:38:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:52 smithi186 ceph-mon[32753]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:53.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:52 smithi111 ceph-mon[27746]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:53.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:52 smithi171 ceph-mon[32337]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 podman[61201]: 2024-04-24 22:38:52.763552009 +0000 UTC m=+0.317335555 container init 61cee8c934343539d754248254ee8aec351cfc5f702b81f7cd57f423c07e76dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 podman[61201]: 2024-04-24 22:38:52.766017079 +0000 UTC m=+0.319800627 container start 61cee8c934343539d754248254ee8aec351cfc5f702b81f7cd57f423c07e76dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 ceph-mds[61219]: -- 172.21.15.171:0/2962246928 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629edb9da40 con 0x5629edb70800 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k[61215]: starting mds.k at 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 bash[61201]: 61cee8c934343539d754248254ee8aec351cfc5f702b81f7cd57f423c07e76dd 2024-04-24T22:38:53.157 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:38:52 smithi171 systemd[1]: Started Ceph mds.k for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:38:53.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 -- 172.21.15.111:0/3767846150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35401027a0 msgr2=0x7f3540104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 --2- 172.21.15.111:0/3767846150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35401027a0 0x7f3540104c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f352c009900 tx=0x7f352c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 -- 172.21.15.111:0/3767846150 shutdown_connections 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 --2- 172.21.15.111:0/3767846150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3540105150 0x7f35401055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 --2- 172.21.15.111:0/3767846150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35401027a0 0x7f3540104c10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 --2- 172.21.15.111:0/3767846150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 0x7f3540102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 -- 172.21.15.111:0/3767846150 >> 172.21.15.111:0/3767846150 conn(0x7f35400fba10 msgr2=0x7f35400fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 -- 172.21.15.111:0/3767846150 shutdown_connections 2024-04-24T22:38:53.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.398+0000 7f3546c00640 1 -- 172.21.15.111:0/3767846150 wait complete. 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.399+0000 7f3546c00640 1 Processor -- start 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.399+0000 7f3546c00640 1 -- start start 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.399+0000 7f3546c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 0x7f354019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.399+0000 7f3546c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3546c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3540105150 0x7f35401a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3546c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113810 con 0x7f3540105150 2024-04-24T22:38:53.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3546c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113690 con 0x7f35401027a0 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3546c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113990 con 0x7f35400ffe30 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52116/0 (socket says 172.21.15.111:52116) 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 -- 172.21.15.111:0/1552912197 learned_addr learned my addr 172.21.15.111:0/1552912197 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3545800640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 0x7f354019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3546200640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3540105150 0x7f35401a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 -- 172.21.15.111:0/1552912197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 msgr2=0x7f354019de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 0x7f354019de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 -- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3540105150 msgr2=0x7f35401a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3540105150 0x7f35401a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 -- 172.21.15.111:0/1552912197 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35401a5aa0 con 0x7f35401027a0 2024-04-24T22:38:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.400+0000 7f3544e00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f352c009230 tx=0x7f352c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:53.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.401+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c03f030 con 0x7f35401027a0 2024-04-24T22:38:53.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.401+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c003ce0 con 0x7f35401027a0 2024-04-24T22:38:53.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.401+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35401a5d30 con 0x7f35401027a0 2024-04-24T22:38:53.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.401+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c035070 con 0x7f35401027a0 2024-04-24T22:38:53.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.401+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a61d0 con 0x7f35401027a0 2024-04-24T22:38:53.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.402+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35400fd500 con 0x7f35401027a0 2024-04-24T22:38:53.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.404+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f352c042030 con 0x7f35401027a0 2024-04-24T22:38:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.405+0000 7f353e000640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 0x7f35280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:53.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.405+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f352c0ef1b0 con 0x7f35401027a0 2024-04-24T22:38:53.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.405+0000 7f3545800640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 0x7f35280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:53.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.406+0000 7f3545800640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 0x7f35280a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f35340095f0 tx=0x7f3534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:53.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.409+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f352c0bc510 con 0x7f35401027a0 2024-04-24T22:38:53.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.652+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3540062340 con 0x7f35401027a0 2024-04-24T22:38:53.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.653+0000 7f353e000640 1 -- 172.21.15.111:0/1552912197 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f352c0bbe10 con 0x7f35401027a0 2024-04-24T22:38:53.655 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:53.656 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T22:38:53.656 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 2 2024-04-24T22:38:53.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 msgr2=0x7f35280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 0x7f35280a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f35340095f0 tx=0x7f3534005f90 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 msgr2=0x7f354019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f352c009230 tx=0x7f352c0042f0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 shutdown_connections 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35280a2b90 0x7f35280a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3540105150 0x7f35401a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35401027a0 0x7f354019e3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 --2- 172.21.15.111:0/1552912197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35400ffe30 0x7f354019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.655+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 >> 172.21.15.111:0/1552912197 conn(0x7f35400fba10 msgr2=0x7f35401008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.656+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 shutdown_connections 2024-04-24T22:38:53.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:53.656+0000 7f3546c00640 1 -- 172.21.15.111:0/1552912197 wait complete. 2024-04-24T22:38:53.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:53.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:53.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:53.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:53.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:53.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:53.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: Deploying daemon mds.j on smithi111 2024-04-24T22:38:53.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:53.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1552912197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:53.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6832/254707808,v1:172.21.15.171:6833/254707808] up:boot 2024-04-24T22:38:53.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: fsmap 2 up:standby 2024-04-24T22:38:53.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:53 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: Deploying daemon mds.j on smithi111 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1552912197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6832/254707808,v1:172.21.15.171:6833/254707808] up:boot 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: fsmap 2 up:standby 2024-04-24T22:38:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:53 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:38:54.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: Deploying daemon mds.j on smithi111 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1552912197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6832/254707808,v1:172.21.15.171:6833/254707808] up:boot 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: fsmap 2 up:standby 2024-04-24T22:38:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:53 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:38:55.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:38:56.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 -- 172.21.15.111:0/3223716326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e88103190 msgr2=0x7f0e88109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:56.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 --2- 172.21.15.111:0/3223716326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e88103190 0x7f0e88109a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009ca0 tx=0x7f0e8002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 -- 172.21.15.111:0/3223716326 shutdown_connections 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 --2- 172.21.15.111:0/3223716326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e88103190 0x7f0e88109a20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 --2- 172.21.15.111:0/3223716326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e881027d0 0x7f0e88102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 --2- 172.21.15.111:0/3223716326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 -- 172.21.15.111:0/3223716326 >> 172.21.15.111:0/3223716326 conn(0x7f0e880fcb50 msgr2=0x7f0e880fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 -- 172.21.15.111:0/3223716326 shutdown_connections 2024-04-24T22:38:56.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.108+0000 7f0e8d000640 1 -- 172.21.15.111:0/3223716326 wait complete. 2024-04-24T22:38:56.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.109+0000 7f0e8d000640 1 Processor -- start 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.109+0000 7f0e8d000640 1 -- start start 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 0x7f0e881a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810d2c0 con 0x7f0e881027d0 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810d140 con 0x7f0e881015e0 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e8d000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810d440 con 0x7f0e88103190 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e86a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e86a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41362/0 (socket says 172.21.15.111:41362) 2024-04-24T22:38:56.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e87400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e86a00640 1 -- 172.21.15.111:0/1945196715 learned_addr learned my addr 172.21.15.111:0/1945196715 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.110+0000 7f0e87e00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 0x7f0e881a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 -- 172.21.15.111:0/1945196715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 msgr2=0x7f0e881a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 0x7f0e881a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 -- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 msgr2=0x7f0e88199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 -- 172.21.15.111:0/1945196715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e881a16b0 con 0x7f0e881015e0 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e86a00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87e00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 0x7f0e881a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e87400640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88199a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9c0 tx=0x7f0e7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:56.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800cd80 con 0x7f0e881015e0 2024-04-24T22:38:56.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e78010910 con 0x7f0e881015e0 2024-04-24T22:38:56.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e881a19a0 con 0x7f0e881015e0 2024-04-24T22:38:56.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78018980 con 0x7f0e881015e0 2024-04-24T22:38:56.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.111+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e881a1ef0 con 0x7f0e881015e0 2024-04-24T22:38:56.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.112+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e881a2120 con 0x7f0e881015e0 2024-04-24T22:38:56.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.115+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0e7801f020 con 0x7f0e881015e0 2024-04-24T22:38:56.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.117+0000 7f0e7fe00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 0x7f0e6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:56.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.117+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0e780cecc0 con 0x7f0e881015e0 2024-04-24T22:38:56.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.117+0000 7f0e86a00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 0x7f0e6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:56.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.118+0000 7f0e86a00640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 0x7f0e6c0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0e8819afa0 tx=0x7f0e74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:56.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.121+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e780d5080 con 0x7f0e881015e0 2024-04-24T22:38:56.283 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 systemd[1]: Starting Ceph mds.j for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:38:56.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.389+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e881a22f0 con 0x7f0e881015e0 2024-04-24T22:38:56.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.390+0000 7f0e7fe00640 1 -- 172.21.15.111:0/1945196715 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f0e7809c020 con 0x7f0e881015e0 2024-04-24T22:38:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:56.392 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T22:38:56.393 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2024-04-24T22:38:56.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 msgr2=0x7f0e6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:56.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 0x7f0e6c0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0e8819afa0 tx=0x7f0e74005f90 comp rx=0 tx=0).stop 2024-04-24T22:38:56.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 msgr2=0x7f0e88199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:56.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88199a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9c0 tx=0x7f0e7800ee80 comp rx=0 tx=0).stop 2024-04-24T22:38:56.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 shutdown_connections 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e6c0a2b90 0x7f0e6c0a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e88103190 0x7f0e881a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e881027d0 0x7f0e88199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 --2- 172.21.15.111:0/1945196715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e881015e0 0x7f0e88199a90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 >> 172.21.15.111:0/1945196715 conn(0x7f0e880fcb50 msgr2=0x7f0e88073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.393+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 shutdown_connections 2024-04-24T22:38:56.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:56.394+0000 7f0e8d000640 1 -- 172.21.15.111:0/1945196715 wait complete. 2024-04-24T22:38:56.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:56 smithi111 ceph-mon[27746]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:56.642 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:56 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1945196715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:56.642 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 podman[68685]: 2024-04-24 22:38:56.292571525 +0000 UTC m=+0.032571665 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:38:56.642 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 podman[68685]: 2024-04-24 22:38:56.408978429 +0000 UTC m=+0.148978599 container create 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T22:38:56.642 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 podman[68685]: 2024-04-24 22:38:56.640098753 +0000 UTC m=+0.380098969 container init 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T22:38:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:56 smithi171 ceph-mon[32337]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:56.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:56 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1945196715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:56 smithi186 ceph-mon[32753]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:56 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1945196715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:57.034 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 podman[68685]: 2024-04-24 22:38:56.650512393 +0000 UTC m=+0.390512539 container start 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T22:38:57.034 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 ceph-mds[68706]: -- 172.21.15.111:0/626547541 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d910fe5a40 con 0x55d910fb9000 2024-04-24T22:38:57.034 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j[68702]: starting mds.j at 2024-04-24T22:38:57.034 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 bash[68685]: 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a 2024-04-24T22:38:57.035 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:38:56 smithi111 systemd[1]: Started Ceph mds.j for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:38:58.131 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: Deploying daemon mds.d on smithi111 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6834/425814781,v1:172.21.15.111:6835/425814781] up:boot 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: fsmap 3 up:standby 2024-04-24T22:38:58.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:57 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:38:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: Deploying daemon mds.d on smithi111 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6834/425814781,v1:172.21.15.111:6835/425814781] up:boot 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: fsmap 3 up:standby 2024-04-24T22:38:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:57 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:38:58.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: Deploying daemon mds.d on smithi111 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6834/425814781,v1:172.21.15.111:6835/425814781] up:boot 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: fsmap 3 up:standby 2024-04-24T22:38:58.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:57 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:38:58.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 -- 172.21.15.111:0/2272517984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f90100790 msgr2=0x7f5f90100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2272517984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f90100790 0x7f5f90100bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009900 tx=0x7f5f7c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 -- 172.21.15.111:0/2272517984 shutdown_connections 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2272517984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 0x7f5f901015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2272517984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f90100790 0x7f5f90100bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2272517984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f900ffe30 0x7f5f90100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 -- 172.21.15.111:0/2272517984 >> 172.21.15.111:0/2272517984 conn(0x7f5f900fba10 msgr2=0x7f5f900fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:58.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 -- 172.21.15.111:0/2272517984 shutdown_connections 2024-04-24T22:38:58.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.623+0000 7f5f95e00640 1 -- 172.21.15.111:0/2272517984 wait complete. 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.624+0000 7f5f95e00640 1 Processor -- start 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.624+0000 7f5f95e00640 1 -- start start 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 0x7f5f90199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 0x7f5f901a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9010f450 con 0x7f5f90101130 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9010f2d0 con 0x7f5f900ffe30 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9010f5d0 con 0x7f5f90100790 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f94a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:58.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f94a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57052/0 (socket says 172.21.15.111:57052) 2024-04-24T22:38:58.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f95400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 0x7f5f901a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:58.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f94a00640 1 -- 172.21.15.111:0/2380590320 learned_addr learned my addr 172.21.15.111:0/2380590320 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:38:58.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.625+0000 7f5f8fe00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 0x7f5f90199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 -- 172.21.15.111:0/2380590320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 msgr2=0x7f5f90199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 0x7f5f90199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 -- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 msgr2=0x7f5f901a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 0x7f5f901a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 -- 172.21.15.111:0/2380590320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f901a1630 con 0x7f5f900ffe30 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f8fe00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 0x7f5f90199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f94a00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000db80 tx=0x7f5f8000b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80017070 con 0x7f5f900ffe30 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f80003b70 con 0x7f5f900ffe30 2024-04-24T22:38:58.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f901a1920 con 0x7f5f900ffe30 2024-04-24T22:38:58.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.626+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f800106c0 con 0x7f5f900ffe30 2024-04-24T22:38:58.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.627+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f901a1e90 con 0x7f5f900ffe30 2024-04-24T22:38:58.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.628+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9010ab80 con 0x7f5f900ffe30 2024-04-24T22:38:58.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.630+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5f80004020 con 0x7f5f900ffe30 2024-04-24T22:38:58.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.631+0000 7f5f8d600640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:58.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.631+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5f800cec30 con 0x7f5f900ffe30 2024-04-24T22:38:58.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.631+0000 7f5f8fe00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:58.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.632+0000 7f5f8fe00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 0x7f5f740a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:58.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.634+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5f80020070 con 0x7f5f900ffe30 2024-04-24T22:38:58.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.878+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f9002d170 con 0x7f5f900ffe30 2024-04-24T22:38:58.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.879+0000 7f5f8d600640 1 -- 172.21.15.111:0/2380590320 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f5f8009c010 con 0x7f5f900ffe30 2024-04-24T22:38:58.881 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:38:58.882 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T22:38:58.882 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 4 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 msgr2=0x7f5f740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 0x7f5f740a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c000bc0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 msgr2=0x7f5f90199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000db80 tx=0x7f5f8000b630 comp rx=0 tx=0).stop 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 shutdown_connections 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f90101130 0x7f5f901a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f90100790 0x7f5f90199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 --2- 172.21.15.111:0/2380590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f900ffe30 0x7f5f90199a60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:58.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 >> 172.21.15.111:0/2380590320 conn(0x7f5f900fba10 msgr2=0x7f5f900fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:58.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 shutdown_connections 2024-04-24T22:38:58.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:38:58.882+0000 7f5f95e00640 1 -- 172.21.15.111:0/2380590320 wait complete. 2024-04-24T22:38:59.139 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:58 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2380590320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:59.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:58 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2380590320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:38:59.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:58 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2380590320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:00.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:38:59 smithi186 ceph-mon[32753]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:00.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:38:59 smithi111 ceph-mon[27746]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:00.285 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:38:59 smithi111 systemd[1]: Starting Ceph mds.d for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:00.307 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:38:59 smithi171 ceph-mon[32337]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:00.637 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 podman[69034]: 2024-04-24 22:39:00.201107255 +0000 UTC m=+0.036356372 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:00.638 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 podman[69034]: 2024-04-24 22:39:00.340686058 +0000 UTC m=+0.175935188 container create ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T22:39:00.638 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 podman[69034]: 2024-04-24 22:39:00.635293584 +0000 UTC m=+0.470542762 container init ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T22:39:00.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:00.976 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 podman[69034]: 2024-04-24 22:39:00.647671717 +0000 UTC m=+0.482920858 container start ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T22:39:00.977 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 ceph-mds[69054]: -- 172.21.15.111:0/3189023569 <== mon.0 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559198e7a40 con 0x5559198bac00 2024-04-24T22:39:00.977 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d[69050]: starting mds.d at 2024-04-24T22:39:00.977 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 bash[69034]: ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec 2024-04-24T22:39:00.977 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:39:00 smithi111 systemd[1]: Started Ceph mds.d for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:01.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 -- 172.21.15.111:0/1813143618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 msgr2=0x7f508c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:01.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 --2- 172.21.15.111:0/1813143618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c100ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f507c009900 tx=0x7f507c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 -- 172.21.15.111:0/1813143618 shutdown_connections 2024-04-24T22:39:01.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 --2- 172.21.15.111:0/1813143618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c102250 0x7f508c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 --2- 172.21.15.111:0/1813143618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c101890 0x7f508c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 --2- 172.21.15.111:0/1813143618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c100ac0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 -- 172.21.15.111:0/1813143618 >> 172.21.15.111:0/1813143618 conn(0x7f508c0fbc30 msgr2=0x7f508c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:01.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 -- 172.21.15.111:0/1813143618 shutdown_connections 2024-04-24T22:39:01.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.107+0000 7f5093800640 1 -- 172.21.15.111:0/1813143618 wait complete. 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.108+0000 7f5093800640 1 Processor -- start 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.108+0000 7f5093800640 1 -- start start 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 0x7f508c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c102250 0x7f508c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f508c10c370 con 0x7f508c102250 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f508c10c1f0 con 0x7f508c1006a0 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5093800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f508c10c4f0 con 0x7f508c101890 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57086/0 (socket says 172.21.15.111:57086) 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 -- 172.21.15.111:0/1807942991 learned_addr learned my addr 172.21.15.111:0/1807942991 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5091800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c102250 0x7f508c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5083e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 0x7f508c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:01.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 -- 172.21.15.111:0/1807942991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 msgr2=0x7f508c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 0x7f508c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 -- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c102250 msgr2=0x7f508c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c102250 0x7f508c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5090e00640 1 -- 172.21.15.111:0/1807942991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c1a1910 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.109+0000 7f5083e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 0x7f508c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f5090e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f507c031440 tx=0x7f507c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c03f030 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c003ce0 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c1a1ba0 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c035070 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.110+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c1a20e0 con 0x7f508c1006a0 2024-04-24T22:39:01.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.111+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c1a22b0 con 0x7f508c1006a0 2024-04-24T22:39:01.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.113+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f507c042030 con 0x7f508c1006a0 2024-04-24T22:39:01.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.114+0000 7f508a000640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 0x7f50700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:01.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.114+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f507c0ef160 con 0x7f508c1006a0 2024-04-24T22:39:01.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.114+0000 7f5083e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 0x7f50700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:01.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.115+0000 7f5083e00640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 0x7f50700a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5074009c20 tx=0x7f5074005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:01.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.118+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f507c0bc4c0 con 0x7f508c1006a0 2024-04-24T22:39:01.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.357+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f508c1a2480 con 0x7f508c1006a0 2024-04-24T22:39:01.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.358+0000 7f508a000640 1 -- 172.21.15.111:0/1807942991 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f507c0bbdc0 con 0x7f508c1006a0 2024-04-24T22:39:01.359 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:01.360 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T22:39:01.360 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 5 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.360+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 msgr2=0x7f50700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.360+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 0x7f50700a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5074009c20 tx=0x7f5074005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.360+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 msgr2=0x7f508c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.360+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f507c031440 tx=0x7f507c032ce0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 shutdown_connections 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f50700a2c10 0x7f50700a50d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f508c102250 0x7f508c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508c101890 0x7f508c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 --2- 172.21.15.111:0/1807942991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f508c1006a0 0x7f508c199d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:01.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 >> 172.21.15.111:0/1807942991 conn(0x7f508c0fbc30 msgr2=0x7f508c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:01.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 shutdown_connections 2024-04-24T22:39:01.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:01.361+0000 7f5093800640 1 -- 172.21.15.111:0/1807942991 wait complete. 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: Deploying daemon mds.e on smithi171 2024-04-24T22:39:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6836/2737045039,v1:172.21.15.111:6837/2737045039] up:boot 2024-04-24T22:39:02.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: fsmap 4 up:standby 2024-04-24T22:39:02.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:39:02.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1807942991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:02.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:01 smithi111 ceph-mon[27746]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:02.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: Deploying daemon mds.e on smithi171 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6836/2737045039,v1:172.21.15.111:6837/2737045039] up:boot 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: fsmap 4 up:standby 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1807942991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:01 smithi171 ceph-mon[32337]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:02.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: Deploying daemon mds.e on smithi171 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6836/2737045039,v1:172.21.15.111:6837/2737045039] up:boot 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: fsmap 4 up:standby 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1807942991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:02.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:01 smithi186 ceph-mon[32753]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:03.157 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 systemd[1]: Starting Ceph mds.e for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:03.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:03.492 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 podman[61526]: 2024-04-24 22:39:03.143038454 +0000 UTC m=+0.018268358 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:03.492 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 podman[61526]: 2024-04-24 22:39:03.263938185 +0000 UTC m=+0.139168086 container create b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:39:03.493 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 podman[61526]: 2024-04-24 22:39:03.491415863 +0000 UTC m=+0.366645764 container init b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 -- 172.21.15.111:0/2012434171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650075780 msgr2=0x7fa650073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 --2- 172.21.15.111:0/2012434171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650075780 0x7fa650073bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa640009860 tx=0x7fa64002a340 comp rx=0 tx=0).stop 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 -- 172.21.15.111:0/2012434171 shutdown_connections 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 --2- 172.21.15.111:0/2012434171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa650109650 0x7fa65010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 --2- 172.21.15.111:0/2012434171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6500741a0 0x7fa650074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 --2- 172.21.15.111:0/2012434171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650075780 0x7fa650073bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 -- 172.21.15.111:0/2012434171 >> 172.21.15.111:0/2012434171 conn(0x7fa6500fcd20 msgr2=0x7fa6500ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:03.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 -- 172.21.15.111:0/2012434171 shutdown_connections 2024-04-24T22:39:03.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.597+0000 7fa657200640 1 -- 172.21.15.111:0/2012434171 wait complete. 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 Processor -- start 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 -- start start 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6500741a0 0x7fa650199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa650075780 0x7fa65019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010f5f0 con 0x7fa650075780 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010f470 con 0x7fa650109650 2024-04-24T22:39:03.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.598+0000 7fa657200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010f770 con 0x7fa6500741a0 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57110/0 (socket says 172.21.15.111:57110) 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 -- 172.21.15.111:0/2207931651 learned_addr learned my addr 172.21.15.111:0/2207931651 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa647e00640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa650075780 0x7fa65019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa64fe00640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6500741a0 0x7fa650199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 -- 172.21.15.111:0/2207931651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6500741a0 msgr2=0x7fa650199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6500741a0 0x7fa650199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 -- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa650075780 msgr2=0x7fa65019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa650075780 0x7fa65019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 -- 172.21.15.111:0/2207931651 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6501a1890 con 0x7fa650109650 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa655200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa64800eb10 tx=0x7fa64800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.599+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648004b20 con 0x7fa650109650 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.600+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6501a1b80 con 0x7fa650109650 2024-04-24T22:39:03.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.600+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa648004cc0 con 0x7fa650109650 2024-04-24T22:39:03.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.600+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648010760 con 0x7fa650109650 2024-04-24T22:39:03.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.600+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6501a2120 con 0x7fa650109650 2024-04-24T22:39:03.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.601+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa650108380 con 0x7fa650109650 2024-04-24T22:39:03.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.603+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa648005020 con 0x7fa650109650 2024-04-24T22:39:03.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.604+0000 7fa64d600640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 0x7fa6340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:03.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.604+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa6480cf370 con 0x7fa650109650 2024-04-24T22:39:03.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.604+0000 7fa64fe00640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 0x7fa6340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:03.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.605+0000 7fa64fe00640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 0x7fa6340a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa65019b150 tx=0x7fa640030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:03.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.612+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa64809c6d0 con 0x7fa650109650 2024-04-24T22:39:03.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.868+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa650062370 con 0x7fa650109650 2024-04-24T22:39:03.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.869+0000 7fa64d600640 1 -- 172.21.15.111:0/2207931651 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fa64809bfd0 con 0x7fa650109650 2024-04-24T22:39:03.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:03.871 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T22:39:03.871 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.871+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 msgr2=0x7fa6340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.871+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 0x7fa6340a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa65019b150 tx=0x7fa640030550 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.871+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 msgr2=0x7fa6501a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.871+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa64800eb10 tx=0x7fa64800c2d0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 shutdown_connections 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa6340a2b90 0x7fa6340a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa650109650 0x7fa6501a11f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa650075780 0x7fa65019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 --2- 172.21.15.111:0/2207931651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6500741a0 0x7fa650199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 >> 172.21.15.111:0/2207931651 conn(0x7fa6500fcd20 msgr2=0x7fa650105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 shutdown_connections 2024-04-24T22:39:03.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:03.872+0000 7fa657200640 1 -- 172.21.15.111:0/2207931651 wait complete. 2024-04-24T22:39:03.906 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 podman[61526]: 2024-04-24 22:39:03.494339873 +0000 UTC m=+0.369569777 container start b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:39:03.906 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 ceph-mds[61544]: -- 172.21.15.171:0/2830608676 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56014e347a40 con 0x56014e31b000 2024-04-24T22:39:03.907 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e[61540]: starting mds.e at 2024-04-24T22:39:03.907 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 bash[61526]: b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 2024-04-24T22:39:03.907 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:39:03 smithi171 systemd[1]: Started Ceph mds.e for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6834/4258808378,v1:172.21.15.171:6835/4258808378] up:boot 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: fsmap 5 up:standby 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.638 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: Deploying daemon mds.f on smithi186 2024-04-24T22:39:04.639 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:04 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2207931651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6834/4258808378,v1:172.21.15.171:6835/4258808378] up:boot 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: fsmap 5 up:standby 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: Deploying daemon mds.f on smithi186 2024-04-24T22:39:04.857 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:04 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2207931651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6834/4258808378,v1:172.21.15.171:6835/4258808378] up:boot 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: fsmap 5 up:standby 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: Deploying daemon mds.f on smithi186 2024-04-24T22:39:05.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:04 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2207931651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:05.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:05.999 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:05 smithi186 systemd[1]: Starting Ceph mds.f for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 -- 172.21.15.111:0/99934748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40107bc0 msgr2=0x7f9b40107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 --2- 172.21.15.111:0/99934748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40107bc0 0x7f9b40107fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009900 tx=0x7f9b3402a410 comp rx=0 tx=0).stop 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 -- 172.21.15.111:0/99934748 shutdown_connections 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 --2- 172.21.15.111:0/99934748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b40110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 --2- 172.21.15.111:0/99934748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40108db0 0x7f9b40109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 --2- 172.21.15.111:0/99934748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40107bc0 0x7f9b40107fe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 -- 172.21.15.111:0/99934748 >> 172.21.15.111:0/99934748 conn(0x7f9b40076b80 msgr2=0x7f9b40078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 -- 172.21.15.111:0/99934748 shutdown_connections 2024-04-24T22:39:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.103+0000 7f9b46400640 1 -- 172.21.15.111:0/99934748 wait complete. 2024-04-24T22:39:06.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 Processor -- start 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 -- start start 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40107bc0 0x7f9b4019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b401a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40113830 con 0x7f9b40108db0 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401136b0 con 0x7f9b40109770 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.104+0000 7f9b46400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401139b0 con 0x7f9b40107bc0 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52944/0 (socket says 172.21.15.111:52944) 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 -- 172.21.15.111:0/3614199617 learned_addr learned my addr 172.21.15.111:0/3614199617 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b45000640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40107bc0 0x7f9b4019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b45a00640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b401a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 -- 172.21.15.111:0/3614199617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40107bc0 msgr2=0x7f9b4019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40107bc0 0x7f9b4019df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 -- 172.21.15.111:0/3614199617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 msgr2=0x7f9b401a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b401a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 -- 172.21.15.111:0/3614199617 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b401a5b00 con 0x7f9b40108db0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b45a00640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b401a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.105+0000 7f9b3fe00640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00e9f0 tx=0x7f9b2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.106+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00cd80 con 0x7f9b40108db0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.106+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b401a5df0 con 0x7f9b40108db0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.106+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2c005500 con 0x7f9b40108db0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.106+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c010690 con 0x7f9b40108db0 2024-04-24T22:39:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.106+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a6260 con 0x7f9b40108db0 2024-04-24T22:39:06.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.107+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4010efb0 con 0x7f9b40108db0 2024-04-24T22:39:06.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.110+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9b2c0046c0 con 0x7f9b40108db0 2024-04-24T22:39:06.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.112+0000 7f9b3d600640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 0x7f9b280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:06.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.113+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9b2c0cf2a0 con 0x7f9b40108db0 2024-04-24T22:39:06.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.113+0000 7f9b45000640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 0x7f9b280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:06.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.114+0000 7f9b45000640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 0x7f9b280a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b34005ab0 tx=0x7f9b34000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:06.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.114+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b2c053740 con 0x7f9b40108db0 2024-04-24T22:39:06.297 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 podman[63391]: 2024-04-24 22:39:05.936324237 +0000 UTC m=+0.018240287 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:06.297 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 podman[63391]: 2024-04-24 22:39:06.042244019 +0000 UTC m=+0.124160056 container create 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T22:39:06.297 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 podman[63391]: 2024-04-24 22:39:06.269515535 +0000 UTC m=+0.351431573 container init 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T22:39:06.297 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 podman[63391]: 2024-04-24 22:39:06.272248639 +0000 UTC m=+0.354164679 container start 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T22:39:06.298 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mds[63411]: -- 172.21.15.186:0/233671127 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af89bb7a40 con 0x55af89b8ac00 2024-04-24T22:39:06.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.353+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b4002d3c0 con 0x7f9b40108db0 2024-04-24T22:39:06.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.355+0000 7f9b3d600640 1 -- 172.21.15.111:0/3614199617 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f9b2c09c600 con 0x7f9b40108db0 2024-04-24T22:39:06.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:06.357 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T22:39:06.357 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2024-04-24T22:39:06.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 msgr2=0x7f9b280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:06.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 0x7f9b280a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b34005ab0 tx=0x7f9b34000bc0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 msgr2=0x7f9b4019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00e9f0 tx=0x7f9b2c00edf0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 shutdown_connections 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b280a2b90 0x7f9b280a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b40109770 0x7f9b401a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40108db0 0x7f9b4019e470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 --2- 172.21.15.111:0/3614199617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b40107bc0 0x7f9b4019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.357+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 >> 172.21.15.111:0/3614199617 conn(0x7f9b40076b80 msgr2=0x7f9b4010c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.358+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 shutdown_connections 2024-04-24T22:39:06.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:06.358+0000 7f9b46400640 1 -- 172.21.15.111:0/3614199617 wait complete. 2024-04-24T22:39:06.585 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: starting mds.f at 2024-04-24T22:39:06.585 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 bash[63391]: 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc 2024-04-24T22:39:06.585 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:39:06 smithi186 systemd[1]: Started Ceph mds.f for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3614199617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:06.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:06 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3614199617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:06.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:06 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3614199617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:07.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:07.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:07.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:07.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:07.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:06 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:07 smithi186 ceph-mon[32753]: Deploying daemon mds.c on smithi186 2024-04-24T22:39:07.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:07 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6834/964018737,v1:172.21.15.186:6835/964018737] up:boot 2024-04-24T22:39:07.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:07 smithi186 ceph-mon[32753]: fsmap 6 up:standby 2024-04-24T22:39:07.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:07 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:39:07.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:07 smithi171 ceph-mon[32337]: Deploying daemon mds.c on smithi186 2024-04-24T22:39:07.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:07 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6834/964018737,v1:172.21.15.186:6835/964018737] up:boot 2024-04-24T22:39:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:07 smithi171 ceph-mon[32337]: fsmap 6 up:standby 2024-04-24T22:39:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:07 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:39:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:07 smithi111 ceph-mon[27746]: Deploying daemon mds.c on smithi186 2024-04-24T22:39:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:07 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6834/964018737,v1:172.21.15.186:6835/964018737] up:boot 2024-04-24T22:39:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:07 smithi111 ceph-mon[27746]: fsmap 6 up:standby 2024-04-24T22:39:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:07 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:39:08.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:08.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 -- 172.21.15.111:0/1677745049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc1027c0 msgr2=0x7f76fc102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:08.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 --2- 172.21.15.111:0/1677745049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc1027c0 0x7f76fc102c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009900 tx=0x7f76f002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 -- 172.21.15.111:0/1677745049 shutdown_connections 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 --2- 172.21.15.111:0/1677745049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 0x7f76fc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 --2- 172.21.15.111:0/1677745049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc1027c0 0x7f76fc102c40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 --2- 172.21.15.111:0/1677745049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 -- 172.21.15.111:0/1677745049 >> 172.21.15.111:0/1677745049 conn(0x7f76fc0fcb80 msgr2=0x7f76fc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.590+0000 7f7702e00640 1 -- 172.21.15.111:0/1677745049 shutdown_connections 2024-04-24T22:39:08.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.591+0000 7f7702e00640 1 -- 172.21.15.111:0/1677745049 wait complete. 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.591+0000 7f7702e00640 1 Processor -- start 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 -- start start 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 0x7f76fc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc074070 0x7f76fc074520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d290 con 0x7f76fc074070 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d110 con 0x7f76fc103180 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d410 con 0x7f76fc1015d0 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7701a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:08.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7701a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:58946/0 (socket says 172.21.15.111:58946) 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7701a00640 1 -- 172.21.15.111:0/2193718014 learned_addr learned my addr 172.21.15.111:0/2193718014 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7702400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc074070 0x7f76fc074520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.592+0000 7f7701000640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 0x7f76fc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 -- 172.21.15.111:0/2193718014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 msgr2=0x7f76fc073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 0x7f76fc073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 -- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc074070 msgr2=0x7f76fc074520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc074070 0x7f76fc074520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 -- 172.21.15.111:0/2193718014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc1a1cf0 con 0x7f76fc1015d0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7701a00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9c0 tx=0x7f76ec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00cd80 con 0x7f76fc1015d0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec0035f0 con 0x7f76fc1015d0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc1a1fe0 con 0x7f76fc1015d0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec010690 con 0x7f76fc1015d0 2024-04-24T22:39:08.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.593+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc1a2440 con 0x7f76fc1015d0 2024-04-24T22:39:08.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.594+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc108970 con 0x7f76fc1015d0 2024-04-24T22:39:08.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.596+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f76ec0037a0 con 0x7f76fc1015d0 2024-04-24T22:39:08.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.598+0000 7f76fa000640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 0x7f76e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:08.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.598+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76ec0cefa0 con 0x7f76fc1015d0 2024-04-24T22:39:08.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.598+0000 7f7701000640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 0x7f76e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:08.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.599+0000 7f7701000640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 0x7f76e40a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009230 tx=0x7f76f00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:08.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.604+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76ec09c300 con 0x7f76fc1015d0 2024-04-24T22:39:08.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:08 smithi186 ceph-mon[32753]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:08.750 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:08 smithi186 systemd[1]: Starting Ceph mds.c for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:08.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:08 smithi111 ceph-mon[27746]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:08.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.844+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76fc02d170 con 0x7f76fc1015d0 2024-04-24T22:39:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.845+0000 7f76fa000640 1 -- 172.21.15.111:0/2193718014 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f76ec09bc00 con 0x7f76fc1015d0 2024-04-24T22:39:08.847 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:08.847 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T22:39:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2024-04-24T22:39:08.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.847+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 msgr2=0x7f76e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.847+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 0x7f76e40a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009230 tx=0x7f76f00315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.847+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 msgr2=0x7f76fc0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.847+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9c0 tx=0x7f76ec00ee80 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.847+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 shutdown_connections 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f76e40a2b90 0x7f76e40a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76fc074070 0x7f76fc074520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76fc103180 0x7f76fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 --2- 172.21.15.111:0/2193718014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76fc1015d0 0x7f76fc0754f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 >> 172.21.15.111:0/2193718014 conn(0x7f76fc0fcb80 msgr2=0x7f76fc105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 shutdown_connections 2024-04-24T22:39:08.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:08.848+0000 7f7702e00640 1 -- 172.21.15.111:0/2193718014 wait complete. 2024-04-24T22:39:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:08 smithi171 ceph-mon[32337]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:09.149 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:08 smithi186 podman[63712]: 2024-04-24 22:39:08.878584451 +0000 UTC m=+0.113524610 container create 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T22:39:09.149 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:08 smithi186 podman[63712]: 2024-04-24 22:39:08.783228325 +0000 UTC m=+0.018168505 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:09.149 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 podman[63712]: 2024-04-24 22:39:09.147945641 +0000 UTC m=+0.382885804 container init 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:39:09.499 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 podman[63712]: 2024-04-24 22:39:09.150613776 +0000 UTC m=+0.385553938 container start 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T22:39:09.499 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mds[63731]: -- 172.21.15.186:0/961794838 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1dd327a40 con 0x55b1dd2fb000 2024-04-24T22:39:09.499 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: starting mds.c at 2024-04-24T22:39:09.499 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 bash[63712]: 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db 2024-04-24T22:39:09.500 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:39:09 smithi186 systemd[1]: Started Ceph mds.c for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2193718014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:09.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:09 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2193718014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:09 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:10.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2193718014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:10.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:10.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:10.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:10.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:09 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:10.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:10 smithi171 ceph-mon[32337]: Deploying daemon mds.h on smithi171 2024-04-24T22:39:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:10 smithi171 ceph-mon[32337]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:10 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6836/1876976469,v1:172.21.15.186:6837/1876976469] up:boot 2024-04-24T22:39:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:10 smithi171 ceph-mon[32337]: fsmap 7 up:standby 2024-04-24T22:39:10.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:10 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:39:10.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:10 smithi111 ceph-mon[27746]: Deploying daemon mds.h on smithi171 2024-04-24T22:39:10.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:10 smithi111 ceph-mon[27746]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:10.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:10 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6836/1876976469,v1:172.21.15.186:6837/1876976469] up:boot 2024-04-24T22:39:10.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:10 smithi111 ceph-mon[27746]: fsmap 7 up:standby 2024-04-24T22:39:10.947 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:10 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:39:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:10 smithi186 ceph-mon[32753]: Deploying daemon mds.h on smithi171 2024-04-24T22:39:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:10 smithi186 ceph-mon[32753]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:10 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6836/1876976469,v1:172.21.15.186:6837/1876976469] up:boot 2024-04-24T22:39:11.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:10 smithi186 ceph-mon[32753]: fsmap 7 up:standby 2024-04-24T22:39:11.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:10 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:39:11.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 -- 172.21.15.111:0/1892355281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f38601039c0 msgr2=0x7f3860103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:11.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 --2- 172.21.15.111:0/1892355281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f38601039c0 0x7f3860103de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3850009900 tx=0x7f385002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 -- 172.21.15.111:0/1892355281 shutdown_connections 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 --2- 172.21.15.111:0/1892355281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860105570 0x7f386010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 --2- 172.21.15.111:0/1892355281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860104bb0 0x7f3860105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 --2- 172.21.15.111:0/1892355281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f38601039c0 0x7f3860103de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 -- 172.21.15.111:0/1892355281 >> 172.21.15.111:0/1892355281 conn(0x7f38600fef30 msgr2=0x7f3860101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 -- 172.21.15.111:0/1892355281 shutdown_connections 2024-04-24T22:39:11.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.080+0000 7f3864c00640 1 -- 172.21.15.111:0/1892355281 wait complete. 2024-04-24T22:39:11.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.081+0000 7f3864c00640 1 Processor -- start 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 -- start start 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 0x7f3860199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860104bb0 0x7f386019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 0x7f38601a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f386010f690 con 0x7f38601039c0 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f386010f510 con 0x7f3860105570 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3864c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f386010f810 con 0x7f3860104bb0 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f385de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 0x7f3860199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:11.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f385de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 0x7f3860199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52994/0 (socket says 172.21.15.111:52994) 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f385de00640 1 -- 172.21.15.111:0/1161062114 learned_addr learned my addr 172.21.15.111:0/1161062114 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f385e800640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 0x7f38601a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.082+0000 7f3855400640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860104bb0 0x7f386019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 -- 172.21.15.111:0/1161062114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860104bb0 msgr2=0x7f386019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860104bb0 0x7f386019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 -- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 msgr2=0x7f3860199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 0x7f3860199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 -- 172.21.15.111:0/1161062114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38601a1910 con 0x7f3860105570 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f385e800640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 0x7f38601a1270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3858009fd0 tx=0x7f385800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800ce20 con 0x7f3860105570 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3858005020 con 0x7f3860105570 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38601a1c00 con 0x7f3860105570 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.083+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858010720 con 0x7f3860105570 2024-04-24T22:39:11.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.084+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38601a21a0 con 0x7f3860105570 2024-04-24T22:39:11.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.085+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386010ace0 con 0x7f3860105570 2024-04-24T22:39:11.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.086+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3858005540 con 0x7f3860105570 2024-04-24T22:39:11.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.088+0000 7f3857400640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:11.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.088+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38580cf250 con 0x7f3860105570 2024-04-24T22:39:11.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.088+0000 7f385de00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:11.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.089+0000 7f385de00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 0x7f38440a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f38500315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:11.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.092+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f385809c5b0 con 0x7f3860105570 2024-04-24T22:39:11.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.328+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3860062340 con 0x7f3860105570 2024-04-24T22:39:11.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.329+0000 7f3857400640 1 -- 172.21.15.111:0/1161062114 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f385809beb0 con 0x7f3860105570 2024-04-24T22:39:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:11.331 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T22:39:11.331 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 8 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 msgr2=0x7f38440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 0x7f38440a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f38500315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 msgr2=0x7f38601a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 0x7f38601a1270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3858009fd0 tx=0x7f385800ef10 comp rx=0 tx=0).stop 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 shutdown_connections 2024-04-24T22:39:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3860105570 0x7f38601a1270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3860104bb0 0x7f386019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 --2- 172.21.15.111:0/1161062114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38601039c0 0x7f3860199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 >> 172.21.15.111:0/1161062114 conn(0x7f38600fef30 msgr2=0x7f3860107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 shutdown_connections 2024-04-24T22:39:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:11.332+0000 7f3864c00640 1 -- 172.21.15.111:0/1161062114 wait complete. 2024-04-24T22:39:11.572 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 systemd[1]: Starting Ceph mds.h for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:11.896 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1161062114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 podman[61837]: 2024-04-24 22:39:11.473260109 +0000 UTC m=+0.018047484 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 podman[61837]: 2024-04-24 22:39:11.583294666 +0000 UTC m=+0.128082049 container create 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 podman[61837]: 2024-04-24 22:39:11.785916948 +0000 UTC m=+0.330704299 container init 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 podman[61837]: 2024-04-24 22:39:11.78888681 +0000 UTC m=+0.333674151 container start 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 ceph-mds[61856]: -- 172.21.15.171:0/470924231 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b8b437a40 con 0x559b8b40ac00 2024-04-24T22:39:11.915 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: starting mds.h at 2024-04-24T22:39:11.916 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 bash[61837]: 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff 2024-04-24T22:39:11.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1161062114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:12.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1161062114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:12.156 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:39:11 smithi171 systemd[1]: Started Ceph mds.h for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:12.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:12 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:12.884 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:12.884 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.884 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:12.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:12.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:12 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:12 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:13.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:13.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 -- 172.21.15.111:0/3471101283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 msgr2=0x7ff524109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 --2- 172.21.15.111:0/3471101283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff524109230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 -- 172.21.15.111:0/3471101283 shutdown_connections 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 --2- 172.21.15.111:0/3471101283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524109770 0x7ff524110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 --2- 172.21.15.111:0/3471101283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff524109230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 --2- 172.21.15.111:0/3471101283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524107bc0 0x7ff524107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 -- 172.21.15.111:0/3471101283 >> 172.21.15.111:0/3471101283 conn(0x7ff524076b80 msgr2=0x7ff524078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.557+0000 7ff52b200640 1 -- 172.21.15.111:0/3471101283 shutdown_connections 2024-04-24T22:39:13.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.558+0000 7ff52b200640 1 -- 172.21.15.111:0/3471101283 wait complete. 2024-04-24T22:39:13.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.558+0000 7ff52b200640 1 Processor -- start 2024-04-24T22:39:13.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 -- start start 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff52419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 0x7ff5241a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff524113870 con 0x7ff524108db0 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241136f0 con 0x7ff524107bc0 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241139f0 con 0x7ff524109770 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff529400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff52419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff529e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff529e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44032/0 (socket says 172.21.15.111:44032) 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff529e00640 1 -- 172.21.15.111:0/2101803201 learned_addr learned my addr 172.21.15.111:0/2101803201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:13.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.559+0000 7ff52a800640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 0x7ff5241a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 -- 172.21.15.111:0/2101803201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 msgr2=0x7ff5241a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 0x7ff5241a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 -- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 msgr2=0x7ff52419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff52419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 -- 172.21.15.111:0/2101803201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5241a5ad0 con 0x7ff524107bc0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff52a800640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 0x7ff5241a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff529e00640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d770 tx=0x7ff51800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff524107bc0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5241a5dc0 con 0x7ff524107bc0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff518009c30 con 0x7ff524107bc0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518003dd0 con 0x7ff524107bc0 2024-04-24T22:39:13.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.560+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5241a6220 con 0x7ff524107bc0 2024-04-24T22:39:13.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.561+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52410eee0 con 0x7ff524107bc0 2024-04-24T22:39:13.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.564+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff518004000 con 0x7ff524107bc0 2024-04-24T22:39:13.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.565+0000 7ff51ea00640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 0x7ff50c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:13.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.565+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5180ce9e0 con 0x7ff524107bc0 2024-04-24T22:39:13.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.565+0000 7ff529400640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 0x7ff50c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:13.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.566+0000 7ff529400640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 0x7ff50c0a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff5100315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:13.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.567+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff518009de0 con 0x7ff524107bc0 2024-04-24T22:39:13.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:13 smithi111 ceph-mon[27746]: Deploying daemon mds.i on smithi186 2024-04-24T22:39:13.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6836/3780798137,v1:172.21.15.171:6837/3780798137] up:boot 2024-04-24T22:39:13.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:13 smithi111 ceph-mon[27746]: fsmap 8 up:standby 2024-04-24T22:39:13.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:13 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:39:13.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.806+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff52402d170 con 0x7ff524107bc0 2024-04-24T22:39:13.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.807+0000 7ff51ea00640 1 -- 172.21.15.111:0/2101803201 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7ff518097070 con 0x7ff524107bc0 2024-04-24T22:39:13.810 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:13.810 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T22:39:13.810 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-04-24T22:39:13.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.810+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 msgr2=0x7ff50c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:13.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.810+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 0x7ff50c0a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff5100315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.810+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 msgr2=0x7ff52419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:13.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.810+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d770 tx=0x7ff51800dc30 comp rx=0 tx=0).stop 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 shutdown_connections 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff50c0a2b90 0x7ff50c0a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff524109770 0x7ff5241a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff524108db0 0x7ff52419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 --2- 172.21.15.111:0/2101803201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524107bc0 0x7ff52419deb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:13.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 >> 172.21.15.111:0/2101803201 conn(0x7ff524076b80 msgr2=0x7ff52410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:13.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 shutdown_connections 2024-04-24T22:39:13.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:13.811+0000 7ff52b200640 1 -- 172.21.15.111:0/2101803201 wait complete. 2024-04-24T22:39:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:13 smithi171 ceph-mon[32337]: Deploying daemon mds.i on smithi186 2024-04-24T22:39:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6836/3780798137,v1:172.21.15.171:6837/3780798137] up:boot 2024-04-24T22:39:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:13 smithi171 ceph-mon[32337]: fsmap 8 up:standby 2024-04-24T22:39:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:13 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:39:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:13 smithi186 ceph-mon[32753]: Deploying daemon mds.i on smithi186 2024-04-24T22:39:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6836/3780798137,v1:172.21.15.171:6837/3780798137] up:boot 2024-04-24T22:39:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:13 smithi186 ceph-mon[32753]: fsmap 8 up:standby 2024-04-24T22:39:14.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:13 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:39:14.407 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 systemd[1]: Starting Ceph mds.i for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:14.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:14 smithi186 ceph-mon[32753]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:14.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:14 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2101803201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:14.685 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 podman[64020]: 2024-04-24 22:39:14.308694844 +0000 UTC m=+0.018004175 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:14.685 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 podman[64020]: 2024-04-24 22:39:14.448262841 +0000 UTC m=+0.157572205 container create 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:39:14.685 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 podman[64020]: 2024-04-24 22:39:14.683821185 +0000 UTC m=+0.393130507 container init 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:39:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:14 smithi171 ceph-mon[32337]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:14.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:14 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2101803201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:14.999 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 podman[64020]: 2024-04-24 22:39:14.686808443 +0000 UTC m=+0.396117775 container start 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T22:39:15.000 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 ceph-mds[64039]: -- 172.21.15.186:0/2185788694 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561283afda40 con 0x561283ad0c00 2024-04-24T22:39:15.000 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: starting mds.i at 2024-04-24T22:39:15.000 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 bash[64020]: 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 2024-04-24T22:39:15.000 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:39:14 smithi186 systemd[1]: Started Ceph mds.i for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:15.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:14 smithi111 ceph-mon[27746]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:15.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:14 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2101803201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:15.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:16.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 -- 172.21.15.111:0/4077626230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 msgr2=0x7f9a38109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:16.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 --2- 172.21.15.111:0/4077626230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a38109230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009900 tx=0x7f9a2002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 -- 172.21.15.111:0/4077626230 shutdown_connections 2024-04-24T22:39:16.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 --2- 172.21.15.111:0/4077626230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38109770 0x7f9a38110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 --2- 172.21.15.111:0/4077626230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a38109230 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 --2- 172.21.15.111:0/4077626230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38107bc0 0x7f9a38107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 -- 172.21.15.111:0/4077626230 >> 172.21.15.111:0/4077626230 conn(0x7f9a38076b80 msgr2=0x7f9a38078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 -- 172.21.15.111:0/4077626230 shutdown_connections 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.033+0000 7f9a37e00640 1 -- 172.21.15.111:0/4077626230 wait complete. 2024-04-24T22:39:16.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.034+0000 7f9a37e00640 1 Processor -- start 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 -- start start 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38107bc0 0x7f9a3819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38109770 0x7f9a381a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38113840 con 0x7f9a38108db0 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381136c0 con 0x7f9a38109770 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381139c0 con 0x7f9a38107bc0 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a2e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a2e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34158/0 (socket says 172.21.15.111:34158) 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a2e000640 1 -- 172.21.15.111:0/521419340 learned_addr learned my addr 172.21.15.111:0/521419340 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a36a00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38107bc0 0x7f9a3819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.035+0000 7f9a37400640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38109770 0x7f9a381a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 -- 172.21.15.111:0/521419340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38107bc0 msgr2=0x7f9a3819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38107bc0 0x7f9a3819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 -- 172.21.15.111:0/521419340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38109770 msgr2=0x7f9a381a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38109770 0x7f9a381a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 -- 172.21.15.111:0/521419340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a381a5ae0 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2e000640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009230 tx=0x7f9a200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2003f030 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a20003ce0 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20035070 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a381a5d70 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.036+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a381a6250 con 0x7f9a38108db0 2024-04-24T22:39:16.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.038+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3810ef60 con 0x7f9a38108db0 2024-04-24T22:39:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.040+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9a20042030 con 0x7f9a38108db0 2024-04-24T22:39:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.041+0000 7f9a2fe00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 0x7f9a1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.042+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9a200ef460 con 0x7f9a38108db0 2024-04-24T22:39:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.042+0000 7f9a36a00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 0x7f9a1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:16.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.042+0000 7f9a36a00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 0x7f9a1c0a50d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a38108c10 tx=0x7f9a28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:16.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.044+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9a200bc170 con 0x7f9a38108db0 2024-04-24T22:39:16.117 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.117 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.117 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.117 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:16.117 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: Deploying daemon mds.g on smithi111 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:boot 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: fsmap 9 up:standby 2024-04-24T22:39:16.118 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:15 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: Deploying daemon mds.g on smithi111 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:boot 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: fsmap 9 up:standby 2024-04-24T22:39:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:15 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: Deploying daemon mds.g on smithi111 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:boot 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: fsmap 9 up:standby 2024-04-24T22:39:16.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:15 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:39:16.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.294+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a3802d170 con 0x7f9a38108db0 2024-04-24T22:39:16.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.294+0000 7f9a2fe00640 1 -- 172.21.15.111:0/521419340 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f9a200c0470 con 0x7f9a38108db0 2024-04-24T22:39:16.296 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:16.296 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T22:39:16.297 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 10 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 msgr2=0x7f9a1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 0x7f9a1c0a50d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a38108c10 tx=0x7f9a28005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 msgr2=0x7f9a3819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009230 tx=0x7f9a200042f0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 shutdown_connections 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9a1c0a2c10 0x7f9a1c0a50d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a38109770 0x7f9a381a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a38108db0 0x7f9a3819e400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 --2- 172.21.15.111:0/521419340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38107bc0 0x7f9a3819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:16.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 >> 172.21.15.111:0/521419340 conn(0x7f9a38076b80 msgr2=0x7f9a3810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:16.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 shutdown_connections 2024-04-24T22:39:16.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:16.297+0000 7f9a37e00640 1 -- 172.21.15.111:0/521419340 wait complete. 2024-04-24T22:39:17.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:16 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/521419340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:17.163 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:16 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/521419340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:16 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/521419340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:18.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:17 smithi171 ceph-mon[32337]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:18.217 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:17 smithi111 ceph-mon[27746]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:18.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:17 smithi186 ceph-mon[32753]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:18.317 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:18.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.520+0000 7f9f8b200640 1 -- 172.21.15.111:0/2104959227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 msgr2=0x7f9f84109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.520+0000 7f9f8b200640 1 --2- 172.21.15.111:0/2104959227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84109a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009be0 tx=0x7f9f8002a6c0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.520+0000 7f9f8b200640 1 -- 172.21.15.111:0/2104959227 shutdown_connections 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 --2- 172.21.15.111:0/2104959227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84109a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 --2- 172.21.15.111:0/2104959227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841027d0 0x7f9f84102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 --2- 172.21.15.111:0/2104959227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841015e0 0x7f9f84101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 -- 172.21.15.111:0/2104959227 >> 172.21.15.111:0/2104959227 conn(0x7f9f840fcb50 msgr2=0x7f9f840fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 -- 172.21.15.111:0/2104959227 shutdown_connections 2024-04-24T22:39:18.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.521+0000 7f9f8b200640 1 -- 172.21.15.111:0/2104959227 wait complete. 2024-04-24T22:39:18.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 Processor -- start 2024-04-24T22:39:18.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 -- start start 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841015e0 0x7f9f840754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841027d0 0x7f9f84073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410d260 con 0x7f9f841015e0 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410d0e0 con 0x7f9f841027d0 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.522+0000 7f9f8b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410d3e0 con 0x7f9f84103190 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f89400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841027d0 0x7f9f84073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:58038/0 (socket says 172.21.15.111:58038) 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 -- 172.21.15.111:0/1412887743 learned_addr learned my addr 172.21.15.111:0/1412887743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f89e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841015e0 0x7f9f840754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 -- 172.21.15.111:0/1412887743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841027d0 msgr2=0x7f9f84073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841027d0 0x7f9f84073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 -- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841015e0 msgr2=0x7f9f840754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841015e0 0x7f9f840754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 -- 172.21.15.111:0/1412887743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f84074900 con 0x7f9f84103190 2024-04-24T22:39:18.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.523+0000 7f9f8a800640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009b30 tx=0x7f9f800305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.524+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80041020 con 0x7f9f84103190 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.524+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f80030740 con 0x7f9f84103190 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.524+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f841a1940 con 0x7f9f84103190 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.524+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80036040 con 0x7f9f84103190 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.524+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841a1ee0 con 0x7f9f84103190 2024-04-24T22:39:18.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.525+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f84068480 con 0x7f9f84103190 2024-04-24T22:39:18.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.527+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9f80044020 con 0x7f9f84103190 2024-04-24T22:39:18.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.528+0000 7f9f7ea00640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 0x7f9f6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.528+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9f800ef860 con 0x7f9f84103190 2024-04-24T22:39:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.528+0000 7f9f89e00640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 0x7f9f6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.529+0000 7f9f89e00640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 0x7f9f6c0a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f84102630 tx=0x7f9f78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:18.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.533+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f8004b020 con 0x7f9f84103190 2024-04-24T22:39:18.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.780+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f84062340 con 0x7f9f84103190 2024-04-24T22:39:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.782+0000 7f9f7ea00640 1 -- 172.21.15.111:0/1412887743 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f9f800bcbc0 con 0x7f9f84103190 2024-04-24T22:39:18.785 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:18.785 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T22:39:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 10 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 msgr2=0x7f9f6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 0x7f9f6c0a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f84102630 tx=0x7f9f78005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 msgr2=0x7f9f84074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009b30 tx=0x7f9f800305d0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 shutdown_connections 2024-04-24T22:39:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9f6c0a2c10 0x7f9f6c0a50d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f84103190 0x7f9f84074070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f841027d0 0x7f9f84073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 --2- 172.21.15.111:0/1412887743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f841015e0 0x7f9f840754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.785+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 >> 172.21.15.111:0/1412887743 conn(0x7f9f840fcb50 msgr2=0x7f9f84105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.786+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 shutdown_connections 2024-04-24T22:39:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:18.786+0000 7f9f8b200640 1 -- 172.21.15.111:0/1412887743 wait complete. 2024-04-24T22:39:18.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:18 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1412887743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:18.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:18 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1412887743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:19.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:19.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:18 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:18 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1412887743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:18 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:19.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:18 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:20.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:boot 2024-04-24T22:39:20.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: fsmap 10 up:standby 2024-04-24T22:39:20.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:39:20.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:20.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:20.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:20.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:20.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: Deploying daemon mds.a on smithi111 2024-04-24T22:39:20.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:19 smithi111 ceph-mon[27746]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:boot 2024-04-24T22:39:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: fsmap 10 up:standby 2024-04-24T22:39:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:39:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:20.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:20.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:20.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:20.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: Deploying daemon mds.a on smithi111 2024-04-24T22:39:20.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:19 smithi186 ceph-mon[32753]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:boot 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: fsmap 10 up:standby 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: Deploying daemon mds.a on smithi111 2024-04-24T22:39:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:19 smithi171 ceph-mon[32337]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:20.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.024+0000 7f028be00640 1 -- 172.21.15.111:0/979167203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1049b0 msgr2=0x7f028c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.024+0000 7f028be00640 1 --2- 172.21.15.111:0/979167203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1049b0 0x7f028c104e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0274009900 tx=0x7f027402a410 comp rx=0 tx=0).stop 2024-04-24T22:39:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 -- 172.21.15.111:0/979167203 shutdown_connections 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 --2- 172.21.15.111:0/979167203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c105370 0x7f028c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 --2- 172.21.15.111:0/979167203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1049b0 0x7f028c104e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 --2- 172.21.15.111:0/979167203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c1037c0 0x7f028c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 -- 172.21.15.111:0/979167203 >> 172.21.15.111:0/979167203 conn(0x7f028c0fed50 msgr2=0x7f028c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 -- 172.21.15.111:0/979167203 shutdown_connections 2024-04-24T22:39:21.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 -- 172.21.15.111:0/979167203 wait complete. 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.025+0000 7f028be00640 1 Processor -- start 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 -- start start 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c1049b0 0x7f028c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 0x7f028c1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10f420 con 0x7f028c1037c0 2024-04-24T22:39:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10f2a0 con 0x7f028c105370 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10f5a0 con 0x7f028c1049b0 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34214/0 (socket says 172.21.15.111:34214) 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028aa00640 1 -- 172.21.15.111:0/2469647405 learned_addr learned my addr 172.21.15.111:0/2469647405 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028a000640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c1049b0 0x7f028c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.026+0000 7f028b400640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 0x7f028c1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 -- 172.21.15.111:0/2469647405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c1049b0 msgr2=0x7f028c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c1049b0 0x7f028c199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 -- 172.21.15.111:0/2469647405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 msgr2=0x7f028c1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 0x7f028c1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 -- 172.21.15.111:0/2469647405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c1a16c0 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028b400640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 0x7f028c1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028aa00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f028400ebc0 tx=0x7f028400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284003b70 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0284003d10 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c1a19b0 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284019980 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.027+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a1ed0 con 0x7f028c1037c0 2024-04-24T22:39:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.029+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c0688d0 con 0x7f028c1037c0 2024-04-24T22:39:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.031+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0284019b20 con 0x7f028c1037c0 2024-04-24T22:39:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.032+0000 7f0283400640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 0x7f02700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.032+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f02840cf500 con 0x7f028c1037c0 2024-04-24T22:39:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.032+0000 7f028a000640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 0x7f02700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.033+0000 7f028a000640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 0x7f02700a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02740314e0 tx=0x7f0274000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.036+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f028409c860 con 0x7f028c1037c0 2024-04-24T22:39:21.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.286+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f028c10aae0 con 0x7f028c1037c0 2024-04-24T22:39:21.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.288+0000 7f0283400640 1 -- 172.21.15.111:0/2469647405 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f028409c160 con 0x7f028c1037c0 2024-04-24T22:39:21.291 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:21.291 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-24T22:39:21.291 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 11 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 msgr2=0x7f02700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 0x7f02700a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02740314e0 tx=0x7f0274000bc0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 msgr2=0x7f028c199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f028400ebc0 tx=0x7f028400c6a0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 shutdown_connections 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f02700a2b90 0x7f02700a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f028c105370 0x7f028c1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028c1049b0 0x7f028c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 --2- 172.21.15.111:0/2469647405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f028c1037c0 0x7f028c199aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 >> 172.21.15.111:0/2469647405 conn(0x7f028c0fed50 msgr2=0x7f028c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 shutdown_connections 2024-04-24T22:39:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:21.291+0000 7f028be00640 1 -- 172.21.15.111:0/2469647405 wait complete. 2024-04-24T22:39:21.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:21 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2469647405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:21.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:21 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2469647405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:21.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:21 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2469647405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:22.278 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:21 smithi111 systemd[1]: Starting Ceph mds.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:22.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:22 smithi111 ceph-mon[27746]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:22.534 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 podman[70037]: 2024-04-24 22:39:22.181063059 +0000 UTC m=+0.032421691 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:22.535 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 podman[70037]: 2024-04-24 22:39:22.314670376 +0000 UTC m=+0.166029044 container create d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T22:39:22.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:22 smithi171 ceph-mon[32337]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:22.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:22 smithi186 ceph-mon[32753]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:23.034 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 podman[70037]: 2024-04-24 22:39:22.566161918 +0000 UTC m=+0.417520592 container init d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:39:23.034 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 podman[70037]: 2024-04-24 22:39:22.576449881 +0000 UTC m=+0.427808573 container start d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T22:39:23.035 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 ceph-mds[70056]: -- 172.21.15.111:0/4182554193 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558c4e0da40 con 0x5558c4de0800 2024-04-24T22:39:23.035 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a[70052]: starting mds.a at 2024-04-24T22:39:23.035 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 bash[70037]: d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f 2024-04-24T22:39:23.035 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:39:22 smithi111 systemd[1]: Started Ceph mds.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:23.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 -- 172.21.15.111:0/1295722794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8109950 msgr2=0x7f83c81101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 --2- 172.21.15.111:0/1295722794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8109950 0x7f83c81101e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009be0 tx=0x7f83c402a6d0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 -- 172.21.15.111:0/1295722794 shutdown_connections 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 --2- 172.21.15.111:0/1295722794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8109950 0x7f83c81101e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 --2- 172.21.15.111:0/1295722794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8108f90 0x7f83c8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 --2- 172.21.15.111:0/1295722794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 0x7f83c81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 -- 172.21.15.111:0/1295722794 >> 172.21.15.111:0/1295722794 conn(0x7f83c8076d60 msgr2=0x7f83c8079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 -- 172.21.15.111:0/1295722794 shutdown_connections 2024-04-24T22:39:23.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.520+0000 7f83d0800640 1 -- 172.21.15.111:0/1295722794 wait complete. 2024-04-24T22:39:23.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 Processor -- start 2024-04-24T22:39:23.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 -- start start 2024-04-24T22:39:23.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 0x7f83c819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:23.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8109950 0x7f83c81a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8113a00 con 0x7f83c8109950 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8113880 con 0x7f83c8108f90 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.521+0000 7f83d0800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8113b80 con 0x7f83c8107da0 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83ce800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8109950 0x7f83c81a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:42242/0 (socket says 172.21.15.111:42242) 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 -- 172.21.15.111:0/2032768114 learned_addr learned my addr 172.21.15.111:0/2032768114 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cde00640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 0x7f83c819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 -- 172.21.15.111:0/2032768114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 msgr2=0x7f83c819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 0x7f83c819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 -- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8109950 msgr2=0x7f83c81a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8109950 0x7f83c81a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 -- 172.21.15.111:0/2032768114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c81a5cd0 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.522+0000 7f83cd400640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9c0 tx=0x7f83b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.523+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b800cde0 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.523+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81a5fc0 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.523+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b80035f0 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.523+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8010650 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.523+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81a6480 con 0x7f83c8108f90 2024-04-24T22:39:23.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.524+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c810f060 con 0x7f83c8108f90 2024-04-24T22:39:23.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.526+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f83b80037a0 con 0x7f83c8108f90 2024-04-24T22:39:23.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.527+0000 7f83c2a00640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 0x7f83b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:23.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.528+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f83b80cef40 con 0x7f83c8108f90 2024-04-24T22:39:23.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.528+0000 7f83cde00640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 0x7f83b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:23.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.528+0000 7f83cde00640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 0x7f83b00a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83c8108df0 tx=0x7f83bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:23.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.532+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83b809c2a0 con 0x7f83c8108f90 2024-04-24T22:39:23.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.770+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83c80623e0 con 0x7f83c8108f90 2024-04-24T22:39:23.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.778+0000 7f83c2a00640 1 -- 172.21.15.111:0/2032768114 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f83b809bba0 con 0x7f83c8108f90 2024-04-24T22:39:23.780 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:23.780 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-24T22:39:23.781 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 12 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 msgr2=0x7f83b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 0x7f83b00a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83c8108df0 tx=0x7f83bc005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 msgr2=0x7f83c819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9c0 tx=0x7f83b800ee80 comp rx=0 tx=0).stop 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 shutdown_connections 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f83b00a2b90 0x7f83b00a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c8109950 0x7f83c81a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83c8108f90 0x7f83c819e5f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 --2- 172.21.15.111:0/2032768114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c8107da0 0x7f83c819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:23.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 >> 172.21.15.111:0/2032768114 conn(0x7f83c8076d60 msgr2=0x7f83c810c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:23.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 shutdown_connections 2024-04-24T22:39:23.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:23.781+0000 7f83d0800640 1 -- 172.21.15.111:0/2032768114 wait complete. 2024-04-24T22:39:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:24.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:24.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6840/667255252,v1:172.21.15.111:6841/667255252] up:boot 2024-04-24T22:39:24.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: fsmap 11 up:standby 2024-04-24T22:39:24.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:23 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:39:24.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6840/667255252,v1:172.21.15.111:6841/667255252] up:boot 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: fsmap 11 up:standby 2024-04-24T22:39:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:23 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:39:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6840/667255252,v1:172.21.15.111:6841/667255252] up:boot 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: fsmap 11 up:standby 2024-04-24T22:39:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:23 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:39:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:24 smithi111 ceph-mon[27746]: Deploying daemon mds.b on smithi171 2024-04-24T22:39:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:24 smithi111 ceph-mon[27746]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:24 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2032768114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:25.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:24 smithi171 ceph-mon[32337]: Deploying daemon mds.b on smithi171 2024-04-24T22:39:25.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:24 smithi171 ceph-mon[32337]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:25.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:24 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2032768114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:25.157 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:24 smithi171 systemd[1]: Starting Ceph mds.b for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:39:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:24 smithi186 ceph-mon[32753]: Deploying daemon mds.b on smithi171 2024-04-24T22:39:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:24 smithi186 ceph-mon[32753]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:24 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2032768114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:25.439 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 podman[62148]: 2024-04-24 22:39:25.083368935 +0000 UTC m=+0.017944158 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:39:25.439 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 podman[62148]: 2024-04-24 22:39:25.186076947 +0000 UTC m=+0.120652207 container create 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T22:39:25.439 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 podman[62148]: 2024-04-24 22:39:25.438262161 +0000 UTC m=+0.372837376 container init 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:39:25.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:25.906 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 podman[62148]: 2024-04-24 22:39:25.4410211 +0000 UTC m=+0.375596315 container start 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:39:25.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 ceph-mds[62167]: -- 172.21.15.171:0/3301708925 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56140c8e5a40 con 0x56140c8b8800 2024-04-24T22:39:25.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: starting mds.b at 2024-04-24T22:39:25.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 bash[62148]: 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 2024-04-24T22:39:25.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:25 smithi171 systemd[1]: Started Ceph mds.b for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:39:26.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.011+0000 7fe7f2200640 1 -- 172.21.15.111:0/3247020973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec104bb0 msgr2=0x7fe7ec105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.011+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3247020973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec105030 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 -- 172.21.15.111:0/3247020973 shutdown_connections 2024-04-24T22:39:26.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3247020973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec105570 0x7fe7ec10be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3247020973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec105030 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3247020973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 -- 172.21.15.111:0/3247020973 >> 172.21.15.111:0/3247020973 conn(0x7fe7ec0fef30 msgr2=0x7fe7ec101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:26.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 -- 172.21.15.111:0/3247020973 shutdown_connections 2024-04-24T22:39:26.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.012+0000 7fe7f2200640 1 -- 172.21.15.111:0/3247020973 wait complete. 2024-04-24T22:39:26.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 Processor -- start 2024-04-24T22:39:26.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 -- start start 2024-04-24T22:39:26.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 0x7fe7ec1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10f670 con 0x7fe7ec105570 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10f4f0 con 0x7fe7ec1039c0 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7f2200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10f7f0 con 0x7fe7ec104bb0 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41032/0 (socket says 172.21.15.111:41032) 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7e2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7eb400640 1 -- 172.21.15.111:0/3125177115 learned_addr learned my addr 172.21.15.111:0/3125177115 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.013+0000 7fe7ebe00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 0x7fe7ec1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 -- 172.21.15.111:0/3125177115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 msgr2=0x7fe7ec19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 -- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 msgr2=0x7fe7ec1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 0x7fe7ec1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 -- 172.21.15.111:0/3125177115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1a1910 con 0x7fe7ec1039c0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7e2a00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7ebe00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 0x7fe7ec1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7eb400640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009d20 tx=0x7fe7dc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc010630 con 0x7fe7ec1039c0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc010cb0 con 0x7fe7ec1039c0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec1a1c00 con 0x7fe7ec1039c0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.014+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc018ce0 con 0x7fe7ec1039c0 2024-04-24T22:39:26.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.015+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a21a0 con 0x7fe7ec1039c0 2024-04-24T22:39:26.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.016+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec10ad30 con 0x7fe7ec1039c0 2024-04-24T22:39:26.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.018+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe7dc01f020 con 0x7fe7ec1039c0 2024-04-24T22:39:26.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.020+0000 7fe7e8c00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 0x7fe7d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.020+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7dc0cef30 con 0x7fe7ec1039c0 2024-04-24T22:39:26.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.020+0000 7fe7e2a00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 0x7fe7d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.021+0000 7fe7e2a00640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 0x7fe7d00a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4031440 tx=0x7fe7d40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.024+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7dc09c330 con 0x7fe7ec1039c0 2024-04-24T22:39:26.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.265+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7ec02d380 con 0x7fe7ec1039c0 2024-04-24T22:39:26.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.266+0000 7fe7e8c00640 1 -- 172.21.15.111:0/3125177115 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7fe7dc09bc30 con 0x7fe7ec1039c0 2024-04-24T22:39:26.268 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:26.268 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T22:39:26.268 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 13 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.268+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 msgr2=0x7fe7d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.268+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 0x7fe7d00a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4031440 tx=0x7fe7d40315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 msgr2=0x7fe7ec199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009d20 tx=0x7fe7dc00c6a0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 shutdown_connections 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe7d00a2c10 0x7fe7d00a50d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec105570 0x7fe7ec1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7ec104bb0 0x7fe7ec19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 --2- 172.21.15.111:0/3125177115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7ec1039c0 0x7fe7ec199d00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 >> 172.21.15.111:0/3125177115 conn(0x7fe7ec0fef30 msgr2=0x7fe7ec107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 shutdown_connections 2024-04-24T22:39:26.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.269+0000 7fe7f2200640 1 -- 172.21.15.111:0/3125177115 wait complete. 2024-04-24T22:39:26.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:26.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 -- 172.21.15.111:0/3350749762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 msgr2=0x7f150c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 --2- 172.21.15.111:0/3350749762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c073bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1500009900 tx=0x7f150002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 -- 172.21.15.111:0/3350749762 shutdown_connections 2024-04-24T22:39:26.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 --2- 172.21.15.111:0/3350749762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 0x7f150c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 --2- 172.21.15.111:0/3350749762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 0x7f150c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 --2- 172.21.15.111:0/3350749762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c073bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 -- 172.21.15.111:0/3350749762 >> 172.21.15.111:0/3350749762 conn(0x7f150c0fcb50 msgr2=0x7f150c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:26.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 -- 172.21.15.111:0/3350749762 shutdown_connections 2024-04-24T22:39:26.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.487+0000 7f1513200640 1 -- 172.21.15.111:0/3350749762 wait complete. 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.488+0000 7f1513200640 1 Processor -- start 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 -- start start 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 0x7f150c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 0x7f150c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10f490 con 0x7f150c0741a0 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10f310 con 0x7f150c075780 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1513200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10f610 con 0x7f150c1094a0 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1511400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1511e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 0x7f150c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1511400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41046/0 (socket says 172.21.15.111:41046) 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1511400640 1 -- 172.21.15.111:0/2623588621 learned_addr learned my addr 172.21.15.111:0/2623588621 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.489+0000 7f1512800640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 0x7f150c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 -- 172.21.15.111:0/2623588621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 msgr2=0x7f150c1a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 0x7f150c1a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 -- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 msgr2=0x7f150c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 0x7f150c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 -- 172.21.15.111:0/2623588621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c1a1710 con 0x7f150c075780 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1511400640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f14f800ea60 tx=0x7f14f800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800cde0 con 0x7f150c075780 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f80035f0 con 0x7f150c075780 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8010670 con 0x7f150c075780 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c1a1a00 con 0x7f150c075780 2024-04-24T22:39:26.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.490+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a1f00 con 0x7f150c075780 2024-04-24T22:39:26.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.492+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c1081d0 con 0x7f150c075780 2024-04-24T22:39:26.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.493+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f14f80037a0 con 0x7f150c075780 2024-04-24T22:39:26.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.495+0000 7f1506a00640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 0x7f14f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.495+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f14f80ceeb0 con 0x7f150c075780 2024-04-24T22:39:26.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.495+0000 7f1511e00640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 0x7f14f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.496+0000 7f1511e00640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 0x7f14f40a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1500031440 tx=0x7f15000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.499+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14f809c210 con 0x7f150c075780 2024-04-24T22:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.727+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f150c062340 con 0x7f150c075780 2024-04-24T22:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.729+0000 7f1506a00640 1 -- 172.21.15.111:0/2623588621 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7f14f809bb10 con 0x7f150c075780 2024-04-24T22:39:26.731 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:26.731 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":81,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:38:33.199022+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.731+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 msgr2=0x7f14f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 0x7f14f40a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1500031440 tx=0x7f15000315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 msgr2=0x7f150c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f14f800ea60 tx=0x7f14f800ef20 comp rx=0 tx=0).stop 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 shutdown_connections 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f40a2c10 0x7f14f40a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f150c1094a0 0x7f150c1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f150c075780 0x7f150c19a030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 --2- 172.21.15.111:0/2623588621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c0741a0 0x7f150c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 >> 172.21.15.111:0/2623588621 conn(0x7f150c0fcb50 msgr2=0x7f150c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 shutdown_connections 2024-04-24T22:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.732+0000 7f1513200640 1 -- 172.21.15.111:0/2623588621 wait complete. 2024-04-24T22:39:26.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:39:26.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:39:26.830 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:boot 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: fsmap 12 up:standby 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:26.831 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:26 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3125177115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:26.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:26.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:boot 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: fsmap 12 up:standby 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:26.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:26 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3125177115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 -- 172.21.15.111:0/2438426025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c103200 msgr2=0x7f105c109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 --2- 172.21.15.111:0/2438426025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c103200 0x7f105c109a90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1054009ca0 tx=0x7f105402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 -- 172.21.15.111:0/2438426025 shutdown_connections 2024-04-24T22:39:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 --2- 172.21.15.111:0/2438426025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c103200 0x7f105c109a90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 --2- 172.21.15.111:0/2438426025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 --2- 172.21.15.111:0/2438426025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c101650 0x7f105c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 -- 172.21.15.111:0/2438426025 >> 172.21.15.111:0/2438426025 conn(0x7f105c0fcbe0 msgr2=0x7f105c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 -- 172.21.15.111:0/2438426025 shutdown_connections 2024-04-24T22:39:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.935+0000 7f1063000640 1 -- 172.21.15.111:0/2438426025 wait complete. 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 Processor -- start 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 -- start start 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c103200 0x7f105c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10d300 con 0x7f105c101650 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10d180 con 0x7f105c102840 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.936+0000 7f1063000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10d480 con 0x7f105c103200 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48470/0 (socket says 172.21.15.111:48470) 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1062600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c103200 0x7f105c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061c00640 1 -- 172.21.15.111:0/1906950378 learned_addr learned my addr 172.21.15.111:0/1906950378 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 -- 172.21.15.111:0/1906950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c103200 msgr2=0x7f105c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c103200 0x7f105c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 -- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 msgr2=0x7f105c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 -- 172.21.15.111:0/1906950378 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c1a16f0 con 0x7f105c102840 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061c00640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f1061200640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c19a010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.937+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800cd80 con 0x7f105c102840 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.938+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10480035f0 con 0x7f105c102840 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.938+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048010690 con 0x7f105c102840 2024-04-24T22:39:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.938+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105c1a19e0 con 0x7f105c102840 2024-04-24T22:39:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.938+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105c1a1f30 con 0x7f105c102840 2024-04-24T22:39:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.939+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105c1a21c0 con 0x7f105c102840 2024-04-24T22:39:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.940+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1048003b30 con 0x7f105c102840 2024-04-24T22:39:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.942+0000 7f105a000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 0x7f10440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.942+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f10480cef20 con 0x7f105c102840 2024-04-24T22:39:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.942+0000 7f1061c00640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 0x7f10440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.942+0000 7f1061c00640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 0x7f10440a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10500095f0 tx=0x7f1050005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:26.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:26.946+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f104809c280 con 0x7f105c102840 2024-04-24T22:39:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:boot 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: fsmap 12 up:standby 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:27.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:26 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3125177115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.136+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f105c1a2390 con 0x7f105c102840 2024-04-24T22:39:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.137+0000 7f105a000640 1 -- 172.21.15.111:0/1906950378 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f104809bb80 con 0x7f105c102840 2024-04-24T22:39:27.138 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 msgr2=0x7f10440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 0x7f10440a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10500095f0 tx=0x7f1050005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 msgr2=0x7f105c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c19a010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 shutdown_connections 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f10440a2b90 0x7f10440a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105c103200 0x7f105c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105c102840 0x7f105c19a010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 --2- 172.21.15.111:0/1906950378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105c101650 0x7f105c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 >> 172.21.15.111:0/1906950378 conn(0x7f105c0fcbe0 msgr2=0x7f105c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 shutdown_connections 2024-04-24T22:39:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.139+0000 7f1063000640 1 -- 172.21.15.111:0/1906950378 wait complete. 2024-04-24T22:39:27.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 -- 172.21.15.111:0/2243916855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 msgr2=0x7fb52c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 --2- 172.21.15.111:0/2243916855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c105010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb51802a410 comp rx=0 tx=0).stop 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 -- 172.21.15.111:0/2243916855 shutdown_connections 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 --2- 172.21.15.111:0/2243916855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c105550 0x7fb52c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 --2- 172.21.15.111:0/2243916855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c105010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 --2- 172.21.15.111:0/2243916855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c1039a0 0x7fb52c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 -- 172.21.15.111:0/2243916855 >> 172.21.15.111:0/2243916855 conn(0x7fb52c0fef30 msgr2=0x7fb52c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:27.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 -- 172.21.15.111:0/2243916855 shutdown_connections 2024-04-24T22:39:27.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.349+0000 7fb531000640 1 -- 172.21.15.111:0/2243916855 wait complete. 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 Processor -- start 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 -- start start 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c105550 0x7fb52c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f670 con 0x7fb52c105550 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f4f0 con 0x7fb52c104b90 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb531000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f7f0 con 0x7fb52c1039a0 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb529e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb529e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53450/0 (socket says 172.21.15.111:53450) 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb529e00640 1 -- 172.21.15.111:0/4009482587 learned_addr learned my addr 172.21.15.111:0/4009482587 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb52a800640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c105550 0x7fb52c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.350+0000 7fb529400640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 -- 172.21.15.111:0/4009482587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 msgr2=0x7fb52c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 -- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c105550 msgr2=0x7fb52c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c105550 0x7fb52c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 -- 172.21.15.111:0/4009482587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c1a1880 con 0x7fb52c1039a0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529400640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb529e00640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9c0 tx=0x7fb51c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00cd80 con 0x7fb52c1039a0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb51c0035f0 con 0x7fb52c1039a0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c010690 con 0x7fb52c1039a0 2024-04-24T22:39:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.351+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c1a1b70 con 0x7fb52c1039a0 2024-04-24T22:39:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.352+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a2070 con 0x7fb52c1039a0 2024-04-24T22:39:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.353+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c10ac60 con 0x7fb52c1039a0 2024-04-24T22:39:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.355+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb51c0037a0 con 0x7fb52c1039a0 2024-04-24T22:39:27.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.356+0000 7fb522a00640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 0x7fb5100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.356+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb51c0cf0b0 con 0x7fb52c1039a0 2024-04-24T22:39:27.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.357+0000 7fb529400640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 0x7fb5100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.358+0000 7fb529400640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 0x7fb5100a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:27.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.359+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb51c09c410 con 0x7fb52c1039a0 2024-04-24T22:39:27.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.577+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb52c02d170 con 0x7fb52c1039a0 2024-04-24T22:39:27.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.578+0000 7fb522a00640 1 -- 172.21.15.111:0/4009482587 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fb51c09bd10 con 0x7fb52c1039a0 2024-04-24T22:39:27.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:27.579 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-04-24T22:39:27.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:39:27.580 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-04-24T22:39:27.580 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-04-24T22:39:27.580 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:39:27.580 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-04-24T22:39:27.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.580+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 msgr2=0x7fb5100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.580+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 0x7fb5100a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.580+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 msgr2=0x7fb52c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.580+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9c0 tx=0x7fb51c00ee80 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 shutdown_connections 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb5100a2c10 0x7fb5100a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb52c105550 0x7fb52c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb52c104b90 0x7fb52c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 --2- 172.21.15.111:0/4009482587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1039a0 0x7fb52c199c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 >> 172.21.15.111:0/4009482587 conn(0x7fb52c0fef30 msgr2=0x7fb52c107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 shutdown_connections 2024-04-24T22:39:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.581+0000 7fb531000640 1 -- 172.21.15.111:0/4009482587 wait complete. 2024-04-24T22:39:27.602 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T22:39:27.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T22:39:27.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 -- 172.21.15.111:0/2964064189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce001020d0 msgr2=0x7fce001044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 --2- 172.21.15.111:0/2964064189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce001020d0 0x7fce001044c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009900 tx=0x7fcdf402a410 comp rx=0 tx=0).stop 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 -- 172.21.15.111:0/2964064189 shutdown_connections 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 --2- 172.21.15.111:0/2964064189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce00107330 0x7fce00109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 --2- 172.21.15.111:0/2964064189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 0x7fce00106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 --2- 172.21.15.111:0/2964064189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce001020d0 0x7fce001044c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 -- 172.21.15.111:0/2964064189 >> 172.21.15.111:0/2964064189 conn(0x7fce000fbae0 msgr2=0x7fce000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 -- 172.21.15.111:0/2964064189 shutdown_connections 2024-04-24T22:39:27.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.789+0000 7fce06200640 1 -- 172.21.15.111:0/2964064189 wait complete. 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.790+0000 7fce06200640 1 Processor -- start 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.790+0000 7fce06200640 1 -- start start 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 0x7fce0019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce00107330 0x7fce001a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010d320 con 0x7fce00107330 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010d1a0 con 0x7fce001020d0 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce06200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010d4a0 con 0x7fce00104a00 2024-04-24T22:39:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41092/0 (socket says 172.21.15.111:41092) 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 -- 172.21.15.111:0/433845461 learned_addr learned my addr 172.21.15.111:0/433845461 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce05800640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce00107330 0x7fce001a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fcdffe00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 0x7fce0019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 -- 172.21.15.111:0/433845461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 msgr2=0x7fce0019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 0x7fce0019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 -- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce00107330 msgr2=0x7fce001a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce00107330 0x7fce001a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.791+0000 7fce04e00640 1 -- 172.21.15.111:0/433845461 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce001a1790 con 0x7fce001020d0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fce04e00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4031440 tx=0x7fcdf4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf403f030 con 0x7fce001020d0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce001a1a20 con 0x7fce001020d0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf4003ce0 con 0x7fce001020d0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4035070 con 0x7fce001020d0 2024-04-24T22:39:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.792+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce001a1f10 con 0x7fce001020d0 2024-04-24T22:39:27.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.794+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce00108e80 con 0x7fce001020d0 2024-04-24T22:39:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.795+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcdf4042030 con 0x7fce001020d0 2024-04-24T22:39:27.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.797+0000 7fcdfd600640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 0x7fcde80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:27.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.797+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcdf40ef0d0 con 0x7fce001020d0 2024-04-24T22:39:27.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.797+0000 7fcdffe00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 0x7fcde80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:27.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.797+0000 7fcdffe00640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 0x7fcde80a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce0019b080 tx=0x7fcdec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:27.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:27.805+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcdf40bbde0 con 0x7fce001020d0 2024-04-24T22:39:27.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:27 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2623588621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:27.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:27 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1906950378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:39:27.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:27 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2623588621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:27 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1906950378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:39:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.006+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fce00062340 con 0x7fce001020d0 2024-04-24T22:39:28.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:27 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2623588621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:28.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:27 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1906950378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:39:28.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.641+0000 7fcdfd600640 1 -- 172.21.15.111:0/433845461 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fcdf40c00e0 con 0x7fce001020d0 2024-04-24T22:39:28.644 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 msgr2=0x7fcde80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 0x7fcde80a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce0019b080 tx=0x7fcdec005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 msgr2=0x7fce00199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4031440 tx=0x7fcdf4032ce0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 shutdown_connections 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcde80a2b90 0x7fcde80a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce00107330 0x7fce001a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce00104a00 0x7fce0019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.645+0000 7fce06200640 1 --2- 172.21.15.111:0/433845461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce001020d0 0x7fce00199b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.646+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 >> 172.21.15.111:0/433845461 conn(0x7fce000fbae0 msgr2=0x7fce00101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.646+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 shutdown_connections 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.647+0000 7fce06200640 1 -- 172.21.15.111:0/433845461 wait complete. 2024-04-24T22:39:28.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T22:39:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 -- 172.21.15.111:0/544233846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c103340 msgr2=0x7fe56c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 --2- 172.21.15.111:0/544233846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c103340 0x7fe56c109bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe564009be0 tx=0x7fe56402a6c0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 -- 172.21.15.111:0/544233846 shutdown_connections 2024-04-24T22:39:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 --2- 172.21.15.111:0/544233846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c103340 0x7fe56c109bd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 --2- 172.21.15.111:0/544233846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c102980 0x7fe56c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 --2- 172.21.15.111:0/544233846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c101790 0x7fe56c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 -- 172.21.15.111:0/544233846 >> 172.21.15.111:0/544233846 conn(0x7fe56c0fcd20 msgr2=0x7fe56c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:28.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 -- 172.21.15.111:0/544233846 shutdown_connections 2024-04-24T22:39:28.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.847+0000 7fe572400640 1 -- 172.21.15.111:0/544233846 wait complete. 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.848+0000 7fe572400640 1 Processor -- start 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.848+0000 7fe572400640 1 -- start start 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101790 0x7fe56c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c103340 0x7fe56c1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10d3a0 con 0x7fe56c102980 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10d220 con 0x7fe56c103340 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe572400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10d520 con 0x7fe56c101790 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48528/0 (socket says 172.21.15.111:48528) 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 -- 172.21.15.111:0/1532542815 learned_addr learned my addr 172.21.15.111:0/1532542815 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56b400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101790 0x7fe56c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56be00640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c103340 0x7fe56c1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 -- 172.21.15.111:0/1532542815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101790 msgr2=0x7fe56c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101790 0x7fe56c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 -- 172.21.15.111:0/1532542815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c103340 msgr2=0x7fe56c1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c103340 0x7fe56c1a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.849+0000 7fe56aa00640 1 -- 172.21.15.111:0/1532542815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c1a18c0 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe56aa00640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e9f0 tx=0x7fe55800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55800cd80 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe558010910 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c1a1bb0 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558018990 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.850+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1a2150 con 0x7fe56c102980 2024-04-24T22:39:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.851+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c1a2320 con 0x7fe56c102980 2024-04-24T22:39:28.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.854+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe558010ac0 con 0x7fe56c102980 2024-04-24T22:39:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.855+0000 7fe563e00640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 0x7fe5500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.855+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fe5580cf4c0 con 0x7fe56c102980 2024-04-24T22:39:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.855+0000 7fe56b400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 0x7fe5500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.856+0000 7fe56b400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 0x7fe5500a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19ab50 tx=0x7fe55c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:28.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:28.858+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe55809c6b0 con 0x7fe56c102980 2024-04-24T22:39:28.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:28.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/4009482587' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/433845461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:28.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:28 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/4009482587' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T22:39:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/433845461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:28 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:29.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4009482587' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/433845461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:29.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:28 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T22:39:29.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.067+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe56c1a24f0 con 0x7fe56c102980 2024-04-24T22:39:29.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.641+0000 7fe563e00640 1 -- 172.21.15.111:0/1532542815 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fe55809bfb0 con 0x7fe56c102980 2024-04-24T22:39:29.644 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-04-24T22:39:29.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.646+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 msgr2=0x7fe5500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.646+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 0x7fe5500a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19ab50 tx=0x7fe55c009260 comp rx=0 tx=0).stop 2024-04-24T22:39:29.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.646+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 msgr2=0x7fe56c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.646+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e9f0 tx=0x7fe55800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 shutdown_connections 2024-04-24T22:39:29.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe5500a2c10 0x7fe5500a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c103340 0x7fe56c1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe56c102980 0x7fe56c19a1c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 --2- 172.21.15.111:0/1532542815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101790 0x7fe56c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.650+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 >> 172.21.15.111:0/1532542815 conn(0x7fe56c0fcd20 msgr2=0x7fe56c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.651+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 shutdown_connections 2024-04-24T22:39:29.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.652+0000 7fe572400640 1 -- 172.21.15.111:0/1532542815 wait complete. 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T22:39:29.686 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T22:39:29.687 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.687 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:29.687 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:29 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T22:39:29.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 -- 172.21.15.111:0/1012677390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc101670 msgr2=0x7fddcc101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 --2- 172.21.15.111:0/1012677390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc101670 0x7fddcc101a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fddc000fb10 tx=0x7fddc002e570 comp rx=0 tx=0).stop 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 -- 172.21.15.111:0/1012677390 shutdown_connections 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 --2- 172.21.15.111:0/1012677390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 --2- 172.21.15.111:0/1012677390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc102860 0x7fddcc102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 --2- 172.21.15.111:0/1012677390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc101670 0x7fddcc101a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.858+0000 7fddd3200640 1 -- 172.21.15.111:0/1012677390 >> 172.21.15.111:0/1012677390 conn(0x7fddcc0fcbe0 msgr2=0x7fddcc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.859+0000 7fddd3200640 1 -- 172.21.15.111:0/1012677390 shutdown_connections 2024-04-24T22:39:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.859+0000 7fddd3200640 1 -- 172.21.15.111:0/1012677390 wait complete. 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 Processor -- start 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 -- start start 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 0x7fddcc1956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 0x7fddcc195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc10b160 con 0x7fddcc101670 2024-04-24T22:39:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc10afe0 con 0x7fddcc102860 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd3200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc10b2e0 con 0x7fddcc103220 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 0x7fddcc1956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd2800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 0x7fddcc195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 0x7fddcc1956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48548/0 (socket says 172.21.15.111:48548) 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd2800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53516/0 (socket says 172.21.15.111:53516) 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.860+0000 7fddd1e00640 1 -- 172.21.15.111:0/3631805731 learned_addr learned my addr 172.21.15.111:0/3631805731 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:29.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 -- 172.21.15.111:0/3631805731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 msgr2=0x7fddcc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 -- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 msgr2=0x7fddcc1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 0x7fddcc1956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 -- 172.21.15.111:0/3631805731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc19d300 con 0x7fddcc102860 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd1400640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 0x7fddcc195c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00e9f0 tx=0x7fddbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00ccc0 con 0x7fddcc102860 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc010910 con 0x7fddcc102860 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc19d5f0 con 0x7fddcc102860 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.861+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc018960 con 0x7fddcc102860 2024-04-24T22:39:29.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.862+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc19db90 con 0x7fddcc102860 2024-04-24T22:39:29.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.863+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc108930 con 0x7fddcc102860 2024-04-24T22:39:29.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.864+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fddbc010ac0 con 0x7fddcc102860 2024-04-24T22:39:29.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.866+0000 7fddc6a00640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 0x7fddb40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.866+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fddbc0cf220 con 0x7fddcc102860 2024-04-24T22:39:29.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.866+0000 7fddd1e00640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 0x7fddb40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.867+0000 7fddd1e00640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 0x7fddb40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004d20 tx=0x7fddc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:29.870+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fddbc09bc40 con 0x7fddcc102860 2024-04-24T22:39:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:29.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T22:39:29.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T22:39:29.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:29.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:29 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T22:39:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T22:39:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T22:39:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:29 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T22:39:30.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.111+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fddcc062340 con 0x7fddcc102860 2024-04-24T22:39:30.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.671+0000 7fddc6a00640 1 -- 172.21.15.111:0/3631805731 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fddbc09ff40 con 0x7fddcc102860 2024-04-24T22:39:30.673 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.675+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 msgr2=0x7fddb40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.675+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 0x7fddb40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004d20 tx=0x7fddc0000bc0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.675+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 msgr2=0x7fddcc195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.675+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 0x7fddcc195c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00e9f0 tx=0x7fddbc00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 shutdown_connections 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fddb40a2c10 0x7fddb40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc103220 0x7fddcc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddcc102860 0x7fddcc195c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 --2- 172.21.15.111:0/3631805731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddcc101670 0x7fddcc1956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 >> 172.21.15.111:0/3631805731 conn(0x7fddcc0fcbe0 msgr2=0x7fddcc105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:30.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.676+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 shutdown_connections 2024-04-24T22:39:30.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.677+0000 7fddd3200640 1 -- 172.21.15.111:0/3631805731 wait complete. 2024-04-24T22:39:30.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:30.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 -- 172.21.15.111:0/631372823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 msgr2=0x7f3918101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 --2- 172.21.15.111:0/631372823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f3918101bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02a410 comp rx=0 tx=0).stop 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 -- 172.21.15.111:0/631372823 shutdown_connections 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 --2- 172.21.15.111:0/631372823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3918103360 0x7f3918109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 --2- 172.21.15.111:0/631372823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39181029a0 0x7f3918102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 --2- 172.21.15.111:0/631372823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f3918101bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 -- 172.21.15.111:0/631372823 >> 172.21.15.111:0/631372823 conn(0x7f39180fcd20 msgr2=0x7f39180ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 -- 172.21.15.111:0/631372823 shutdown_connections 2024-04-24T22:39:30.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.872+0000 7f391fc00640 1 -- 172.21.15.111:0/631372823 wait complete. 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.873+0000 7f391fc00640 1 Processor -- start 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.873+0000 7f391fc00640 1 -- start start 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.873+0000 7f391fc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f39180754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.873+0000 7f391fc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.873+0000 7f391fc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 0x7f3918074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391fc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d480 con 0x7f39181017b0 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391fc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d300 con 0x7f3918103360 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391fc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d600 con 0x7f39181029a0 2024-04-24T22:39:30.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f39180754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53524/0 (socket says 172.21.15.111:53524) 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 -- 172.21.15.111:0/1797211982 learned_addr learned my addr 172.21.15.111:0/1797211982 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391dc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 0x7f3918074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 -- 172.21.15.111:0/1797211982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 msgr2=0x7f3918074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 0x7f3918074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 -- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 msgr2=0x7f39180754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f39180754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f3917e00640 1 -- 172.21.15.111:0/1797211982 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3918074900 con 0x7f39181029a0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391dc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 0x7f3918074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.874+0000 7f391d200640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f39180754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f3917e00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f390400e9f0 tx=0x7f390400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390400cd80 con 0x7f39181029a0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39181a1bb0 con 0x7f39181029a0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3904010910 con 0x7f39181029a0 2024-04-24T22:39:30.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904018950 con 0x7f39181029a0 2024-04-24T22:39:30.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.875+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39181a2150 con 0x7f39181029a0 2024-04-24T22:39:30.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.876+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3918108a70 con 0x7f39181029a0 2024-04-24T22:39:30.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.880+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3904010ac0 con 0x7f39181029a0 2024-04-24T22:39:30.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.881+0000 7f3915600640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 0x7f39000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.881+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f39040cf2b0 con 0x7f39181029a0 2024-04-24T22:39:30.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.881+0000 7f391d200640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 0x7f39000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.882+0000 7f391d200640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 0x7f39000a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f390c02aa00 tx=0x7f390c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:30.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:30.885+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f390409c360 con 0x7f39181029a0 2024-04-24T22:39:30.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: pgmap v313: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3631805731' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:30.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:30 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: pgmap v313: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T22:39:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T22:39:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3631805731' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:30 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: pgmap v313: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1532542815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3631805731' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T22:39:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T22:39:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:30 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:31.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.096+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f391802d370 con 0x7f39181029a0 2024-04-24T22:39:31.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.097+0000 7f3915600640 1 -- 172.21.15.111:0/1797211982 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22267 (secure 0 0 0) 0x7f390409bc60 con 0x7f39181029a0 2024-04-24T22:39:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:31.100 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":84,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:39:30.113299+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 msgr2=0x7f39000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 0x7f39000a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f390c02aa00 tx=0x7f390c0315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 msgr2=0x7f3918073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f390400e9f0 tx=0x7f390400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 shutdown_connections 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f39000a2c10 0x7f39000a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3918103360 0x7f3918074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39181029a0 0x7f3918073b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 --2- 172.21.15.111:0/1797211982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39181017b0 0x7f39180754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 >> 172.21.15.111:0/1797211982 conn(0x7f39180fcd20 msgr2=0x7f3918105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 shutdown_connections 2024-04-24T22:39:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.100+0000 7f391fc00640 1 -- 172.21.15.111:0/1797211982 wait complete. 2024-04-24T22:39:31.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 -- 172.21.15.111:0/6837363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29241009b0 msgr2=0x7f2924100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 --2- 172.21.15.111:0/6837363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29241009b0 0x7f2924100e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2910009900 tx=0x7f291002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 -- 172.21.15.111:0/6837363 shutdown_connections 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 --2- 172.21.15.111:0/6837363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924101350 0x7f29241017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 --2- 172.21.15.111:0/6837363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29241009b0 0x7f2924100e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 --2- 172.21.15.111:0/6837363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2924100050 0x7f2924100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 -- 172.21.15.111:0/6837363 >> 172.21.15.111:0/6837363 conn(0x7f29240fbc30 msgr2=0x7f29240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 -- 172.21.15.111:0/6837363 shutdown_connections 2024-04-24T22:39:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.289+0000 7f292ae00640 1 -- 172.21.15.111:0/6837363 wait complete. 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.290+0000 7f292ae00640 1 Processor -- start 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.290+0000 7f292ae00640 1 -- start start 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.290+0000 7f292ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.290+0000 7f292ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29241009b0 0x7f292419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.290+0000 7f292ae00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924101350 0x7f29241a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f292ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f600 con 0x7f2924100050 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f292ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f480 con 0x7f29241009b0 2024-04-24T22:39:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f292ae00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f780 con 0x7f2924101350 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48588/0 (socket says 172.21.15.111:48588) 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29241009b0 0x7f292419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 -- 172.21.15.111:0/106751870 learned_addr learned my addr 172.21.15.111:0/106751870 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2928e00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924101350 0x7f29241a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 -- 172.21.15.111:0/106751870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924101350 msgr2=0x7f29241a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924101350 0x7f29241a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 -- 172.21.15.111:0/106751870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29241009b0 msgr2=0x7f292419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29241009b0 0x7f292419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.291+0000 7f2923e00640 1 -- 172.21.15.111:0/106751870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241a18b0 con 0x7f2924100050 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f2923e00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9f0 tx=0x7f291400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400cd80 con 0x7f2924100050 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914005500 con 0x7f2924100050 2024-04-24T22:39:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914010690 con 0x7f2924100050 2024-04-24T22:39:31.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29241a1ba0 con 0x7f2924100050 2024-04-24T22:39:31.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.292+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a2070 con 0x7f2924100050 2024-04-24T22:39:31.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.293+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292410ada0 con 0x7f2924100050 2024-04-24T22:39:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.296+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f29140046c0 con 0x7f2924100050 2024-04-24T22:39:31.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.298+0000 7f2920c00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 0x7f29080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.299+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f29140cf310 con 0x7f2924100050 2024-04-24T22:39:31.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.299+0000 7f2923400640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 0x7f29080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.299+0000 7f2923400640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 0x7f29080a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f2910031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:31.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.307+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f291409c320 con 0x7f2924100050 2024-04-24T22:39:31.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.538+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29240623e0 con 0x7f2924100050 2024-04-24T22:39:31.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.541+0000 7f2920c00640 1 -- 172.21.15.111:0/106751870 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22267 (secure 0 0 0) 0x7f2914086070 con 0x7f2924100050 2024-04-24T22:39:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:31.543 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":84,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:39:30.113299+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:31.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.543+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 msgr2=0x7f29080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.543+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 0x7f29080a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f2910031400 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.543+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 msgr2=0x7f2924199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.543+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9f0 tx=0x7f291400edf0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 shutdown_connections 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29080a2c10 0x7f29080a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924101350 0x7f29241a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29241009b0 0x7f292419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 --2- 172.21.15.111:0/106751870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2924100050 0x7f2924199c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 >> 172.21.15.111:0/106751870 conn(0x7f29240fbc30 msgr2=0x7f29240fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 shutdown_connections 2024-04-24T22:39:31.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.544+0000 7f292ae00640 1 -- 172.21.15.111:0/106751870 wait complete. 2024-04-24T22:39:31.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T22:39:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 -- 172.21.15.111:0/2705223913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 msgr2=0x7f4278103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 --2- 172.21.15.111:0/2705223913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278103ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 -- 172.21.15.111:0/2705223913 shutdown_connections 2024-04-24T22:39:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 --2- 172.21.15.111:0/2705223913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f427810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 --2- 172.21.15.111:0/2705223913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f4278104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 --2- 172.21.15.111:0/2705223913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278103ca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 -- 172.21.15.111:0/2705223913 >> 172.21.15.111:0/2705223913 conn(0x7f42780fedf0 msgr2=0x7f4278101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 -- 172.21.15.111:0/2705223913 shutdown_connections 2024-04-24T22:39:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.755+0000 7f4277e00640 1 -- 172.21.15.111:0/2705223913 wait complete. 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 Processor -- start 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 -- start start 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f427819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f42781a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f427810f520 con 0x7f4278103880 2024-04-24T22:39:31.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f427810f3a0 con 0x7f4278105430 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.756+0000 7f4277e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f427810f6a0 con 0x7f4278104a70 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48600/0 (socket says 172.21.15.111:48600) 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 -- 172.21.15.111:0/2796857757 learned_addr learned my addr 172.21.15.111:0/2796857757 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276000640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f427819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4277400640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f42781a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 -- 172.21.15.111:0/2796857757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 msgr2=0x7f427819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f427819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 -- 172.21.15.111:0/2796857757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 msgr2=0x7f42781a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f42781a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:31.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 -- 172.21.15.111:0/2796857757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781a17d0 con 0x7f4278103880 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276000640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f427819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4277400640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f42781a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.757+0000 7f4276a00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4268031440 tx=0x7f4268032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.758+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426803f030 con 0x7f4278103880 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.758+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4268003ce0 con 0x7f4278103880 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.758+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781a1a60 con 0x7f4278103880 2024-04-24T22:39:31.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.758+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268035070 con 0x7f4278103880 2024-04-24T22:39:31.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.758+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781a1fa0 con 0x7f4278103880 2024-04-24T22:39:31.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.759+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427810ac40 con 0x7f4278103880 2024-04-24T22:39:31.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.761+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4268042030 con 0x7f4278103880 2024-04-24T22:39:31.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.763+0000 7f426f400640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 0x7f425c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:31.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.763+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f42680ef7d0 con 0x7f4278103880 2024-04-24T22:39:31.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.763+0000 7f4276000640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 0x7f425c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:31.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.764+0000 7f4276000640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 0x7f425c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42600095f0 tx=0x7f4260005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:31.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:31.768+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42680bc7e0 con 0x7f4278103880 2024-04-24T22:39:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T22:39:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T22:39:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T22:39:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1797211982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/106751870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:32.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:31 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.013+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f427802d170 con 0x7f4278103880 2024-04-24T22:39:32.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1797211982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/106751870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:31 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1797211982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/106751870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:31 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:39:32.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.938+0000 7f426f400640 1 -- 172.21.15.111:0/2796857757 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f42680bc0e0 con 0x7f4278103880 2024-04-24T22:39:32.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 msgr2=0x7f425c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 0x7f425c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42600095f0 tx=0x7f4260005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 msgr2=0x7f4278199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4268031440 tx=0x7f4268032ce0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 shutdown_connections 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f425c0a2c10 0x7f425c0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4278105430 0x7f42781a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4278104a70 0x7f427819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 --2- 172.21.15.111:0/2796857757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278103880 0x7f4278199bb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 >> 172.21.15.111:0/2796857757 conn(0x7f42780fedf0 msgr2=0x7f4278107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.942+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 shutdown_connections 2024-04-24T22:39:32.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:32.943+0000 7f4277e00640 1 -- 172.21.15.111:0/2796857757 wait complete. 2024-04-24T22:39:32.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: pgmap v317: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-04-24T22:39:32.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:active 2024-04-24T22:39:32.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:32.962 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:32.962 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T22:39:32.962 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T22:39:32.962 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:32 smithi111 ceph-mon[27746]: Cluster is now healthy 2024-04-24T22:39:32.963 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T22:39:32.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T22:39:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: pgmap v317: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-04-24T22:39:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:active 2024-04-24T22:39:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T22:39:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T22:39:33.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:32 smithi186 ceph-mon[32753]: Cluster is now healthy 2024-04-24T22:39:33.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: pgmap v317: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-04-24T22:39:33.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] up:active 2024-04-24T22:39:33.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:33.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:33.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T22:39:33.212 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T22:39:33.212 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:32 smithi171 ceph-mon[32337]: Cluster is now healthy 2024-04-24T22:39:33.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 -- 172.21.15.111:0/3271029744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c103880 msgr2=0x7fd88c103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 --2- 172.21.15.111:0/3271029744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c103880 0x7fd88c103ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd880009900 tx=0x7fd88002a410 comp rx=0 tx=0).stop 2024-04-24T22:39:33.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 -- 172.21.15.111:0/3271029744 shutdown_connections 2024-04-24T22:39:33.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 --2- 172.21.15.111:0/3271029744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c105430 0x7fd88c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 --2- 172.21.15.111:0/3271029744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c104a70 0x7fd88c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 --2- 172.21.15.111:0/3271029744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c103880 0x7fd88c103ca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 -- 172.21.15.111:0/3271029744 >> 172.21.15.111:0/3271029744 conn(0x7fd88c0fedf0 msgr2=0x7fd88c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 -- 172.21.15.111:0/3271029744 shutdown_connections 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.164+0000 7fd893000640 1 -- 172.21.15.111:0/3271029744 wait complete. 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 Processor -- start 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 -- start start 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103880 0x7fd88c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 0x7fd88c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10f520 con 0x7fd88c104a70 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10f3a0 con 0x7fd88c105430 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.165+0000 7fd893000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10f6a0 con 0x7fd88c103880 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd891200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd891200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48618/0 (socket says 172.21.15.111:48618) 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 0x7fd88c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd891200640 1 -- 172.21.15.111:0/2819642838 learned_addr learned my addr 172.21.15.111:0/2819642838 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:33.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd891c00640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103880 0x7fd88c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 -- 172.21.15.111:0/2819642838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103880 msgr2=0x7fd88c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103880 0x7fd88c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 -- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 msgr2=0x7fd88c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 -- 172.21.15.111:0/2819642838 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c1a17e0 con 0x7fd88c105430 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd891200640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.166+0000 7fd892600640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 0x7fd88c1a1170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd88400ec50 tx=0x7fd88400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.167+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884003b70 con 0x7fd88c105430 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.167+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884003d10 con 0x7fd88c105430 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.167+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a1ad0 con 0x7fd88c105430 2024-04-24T22:39:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.167+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884010890 con 0x7fd88c105430 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.167+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a1fd0 con 0x7fd88c105430 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.168+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c10aba0 con 0x7fd88c105430 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.170+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd884003fc0 con 0x7fd88c105430 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.171+0000 7fd88a000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 0x7fd8740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.171+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd8840cf550 con 0x7fd88c105430 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.171+0000 7fd891c00640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 0x7fd8740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.172+0000 7fd891c00640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 0x7fd8740a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd88002aa00 tx=0x7fd8800315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.176+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd88409c560 con 0x7fd88c105430 2024-04-24T22:39:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.414+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fd88c062340 con 0x7fd88c105430 2024-04-24T22:39:33.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.953+0000 7fd88a000640 1 -- 172.21.15.111:0/2819642838 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fd88409be60 con 0x7fd88c105430 2024-04-24T22:39:33.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 msgr2=0x7fd8740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 0x7fd8740a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd88002aa00 tx=0x7fd8800315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 msgr2=0x7fd88c1a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 0x7fd88c1a1170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd88400ec50 tx=0x7fd88400c6a0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 shutdown_connections 2024-04-24T22:39:33.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd8740a2b90 0x7fd8740a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd88c105430 0x7fd88c1a1170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd88c104a70 0x7fd88c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 --2- 172.21.15.111:0/2819642838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103880 0x7fd88c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 >> 172.21.15.111:0/2819642838 conn(0x7fd88c0fedf0 msgr2=0x7fd88c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 shutdown_connections 2024-04-24T22:39:33.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:33.958+0000 7fd893000640 1 -- 172.21.15.111:0/2819642838 wait complete. 2024-04-24T22:39:33.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T22:39:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 -- 172.21.15.111:0/1629317164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc074140 msgr2=0x7f6cfc0745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 --2- 172.21.15.111:0/1629317164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc074140 0x7f6cfc0745a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009900 tx=0x7f6ce402a480 comp rx=0 tx=0).stop 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 -- 172.21.15.111:0/1629317164 shutdown_connections 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 --2- 172.21.15.111:0/1629317164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc109620 0x7f6cfc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 --2- 172.21.15.111:0/1629317164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc074140 0x7f6cfc0745a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 --2- 172.21.15.111:0/1629317164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc075720 0x7f6cfc073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 -- 172.21.15.111:0/1629317164 >> 172.21.15.111:0/1629317164 conn(0x7f6cfc0fcd20 msgr2=0x7f6cfc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 -- 172.21.15.111:0/1629317164 shutdown_connections 2024-04-24T22:39:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.159+0000 7f6d02a00640 1 -- 172.21.15.111:0/1629317164 wait complete. 2024-04-24T22:39:34.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.160+0000 7f6d02a00640 1 Processor -- start 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.160+0000 7f6d02a00640 1 -- start start 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc074140 0x7f6cfc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc109620 0x7f6cfc1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10f660 con 0x7f6cfc075720 2024-04-24T22:39:34.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10f4e0 con 0x7f6cfc109620 2024-04-24T22:39:34.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d02a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10f7e0 con 0x7f6cfc074140 2024-04-24T22:39:34.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48628/0 (socket says 172.21.15.111:48628) 2024-04-24T22:39:34.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 -- 172.21.15.111:0/848904209 learned_addr learned my addr 172.21.15.111:0/848904209 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cfbe00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc074140 0x7f6cfc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6d00a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc109620 0x7f6cfc1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 -- 172.21.15.111:0/848904209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc074140 msgr2=0x7f6cfc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc074140 0x7f6cfc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 -- 172.21.15.111:0/848904209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc109620 msgr2=0x7f6cfc1a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc109620 0x7f6cfc1a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.161+0000 7f6cf3400640 1 -- 172.21.15.111:0/848904209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc1a18f0 con 0x7f6cfc075720 2024-04-24T22:39:34.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6cf3400640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4031440 tx=0x7f6ce4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce403f030 con 0x7f6cfc075720 2024-04-24T22:39:34.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4003ce0 con 0x7f6cfc075720 2024-04-24T22:39:34.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4035070 con 0x7f6cfc075720 2024-04-24T22:39:34.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc1a1b80 con 0x7f6cfc075720 2024-04-24T22:39:34.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.162+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1a2020 con 0x7f6cfc075720 2024-04-24T22:39:34.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.163+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc1083e0 con 0x7f6cfc075720 2024-04-24T22:39:34.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.165+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6ce4042030 con 0x7f6cfc075720 2024-04-24T22:39:34.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.166+0000 7f6cf9600640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 0x7f6ce00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.167+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6ce40ef3e0 con 0x7f6cfc075720 2024-04-24T22:39:34.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.167+0000 7f6cfbe00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 0x7f6ce00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.167+0000 7f6cfbe00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 0x7f6ce00a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc19ab80 tx=0x7f6cec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.170+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6ce40bbda0 con 0x7f6cfc075720 2024-04-24T22:39:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:33 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T22:39:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:33 smithi186 ceph-mon[32753]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:33 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2819642838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:33 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:33 smithi186 ceph-mon[32753]: pgmap v318: 97 pgs: 15 unknown, 82 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-24T22:39:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:33 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T22:39:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:33 smithi111 ceph-mon[27746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:33 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2819642838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:33 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:33 smithi111 ceph-mon[27746]: pgmap v318: 97 pgs: 15 unknown, 82 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-24T22:39:34.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:33 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2796857757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T22:39:34.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:33 smithi171 ceph-mon[32337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:34.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:33 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2819642838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:33 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T22:39:34.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:33 smithi171 ceph-mon[32337]: pgmap v318: 97 pgs: 15 unknown, 82 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-24T22:39:34.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.406+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6cfc0623e0 con 0x7f6cfc075720 2024-04-24T22:39:34.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.965+0000 7f6cf9600640 1 -- 172.21.15.111:0/848904209 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f6ce40c00a0 con 0x7f6cfc075720 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 msgr2=0x7f6ce00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 0x7f6ce00a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc19ab80 tx=0x7f6cec009500 comp rx=0 tx=0).stop 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 msgr2=0x7f6cfc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4031440 tx=0x7f6ce4032ce0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 shutdown_connections 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6ce00a2b90 0x7f6ce00a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cfc109620 0x7f6cfc1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc075720 0x7f6cfc19a1f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 --2- 172.21.15.111:0/848904209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cfc074140 0x7f6cfc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 >> 172.21.15.111:0/848904209 conn(0x7f6cfc0fcd20 msgr2=0x7f6cfc1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:34.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.969+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 shutdown_connections 2024-04-24T22:39:34.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:34.970+0000 7f6d02a00640 1 -- 172.21.15.111:0/848904209 wait complete. 2024-04-24T22:39:34.998 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T22:39:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 -- 172.21.15.111:0/2910681190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 msgr2=0x7f27b80777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 --2- 172.21.15.111:0/2910681190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b80777a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009900 tx=0x7f27ac02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 -- 172.21.15.111:0/2910681190 shutdown_connections 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 --2- 172.21.15.111:0/2910681190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 0x7f27b8078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 --2- 172.21.15.111:0/2910681190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b80777a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 --2- 172.21.15.111:0/2910681190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 0x7f27b8076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 -- 172.21.15.111:0/2910681190 >> 172.21.15.111:0/2910681190 conn(0x7f27b80fef30 msgr2=0x7f27b8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.185+0000 7f27c0400640 1 -- 172.21.15.111:0/2910681190 shutdown_connections 2024-04-24T22:39:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.186+0000 7f27c0400640 1 -- 172.21.15.111:0/2910681190 wait complete. 2024-04-24T22:39:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.186+0000 7f27c0400640 1 Processor -- start 2024-04-24T22:39:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 -- start start 2024-04-24T22:39:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 0x7f27b819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 0x7f27b81a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8113a30 con 0x7f27b8077ce0 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81138b0 con 0x7f27b8077340 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27c0400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8113bb0 con 0x7f27b8076100 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27be400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 0x7f27b81a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27bd000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27bd000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41012/0 (socket says 172.21.15.111:41012) 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27bd000640 1 -- 172.21.15.111:0/1811060575 learned_addr learned my addr 172.21.15.111:0/1811060575 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.187+0000 7f27bda00640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 0x7f27b819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 -- 172.21.15.111:0/1811060575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 msgr2=0x7f27b819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 0x7f27b819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 -- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 msgr2=0x7f27b81a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 0x7f27b81a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 -- 172.21.15.111:0/1811060575 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81a5cf0 con 0x7f27b8077340 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27bd000640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27ac031440 tx=0x7f27ac032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac03f030 con 0x7f27b8077340 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac003ce0 con 0x7f27b8077340 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b81a5f80 con 0x7f27b8077340 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac035070 con 0x7f27b8077340 2024-04-24T22:39:35.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.188+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a63f0 con 0x7f27b8077340 2024-04-24T22:39:35.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.190+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8110310 con 0x7f27b8077340 2024-04-24T22:39:35.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.191+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f27ac042030 con 0x7f27b8077340 2024-04-24T22:39:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.193+0000 7f27b6000640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 0x7f27a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.193+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f27ac0ef4a0 con 0x7f27b8077340 2024-04-24T22:39:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.193+0000 7f27bda00640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 0x7f27a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.194+0000 7f27bda00640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 0x7f27a00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005d00 tx=0x7f27a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:35.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.197+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27ac0bbe60 con 0x7f27b8077340 2024-04-24T22:39:35.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T22:39:35.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T22:39:35.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T22:39:35.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:35.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-24T22:39:35.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-24T22:39:35.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:34 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-24T22:39:35.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:34 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T22:39:35.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-24T22:39:35.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:34 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T22:39:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.437+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f27b80623e0 con 0x7f27b8077340 2024-04-24T22:39:35.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.988+0000 7f27b6000640 1 -- 172.21.15.111:0/1811060575 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f27ac0c0160 con 0x7f27b8077340 2024-04-24T22:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 msgr2=0x7f27a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 0x7f27a00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005d00 tx=0x7f27a400b410 comp rx=0 tx=0).stop 2024-04-24T22:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 msgr2=0x7f27b819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27ac031440 tx=0x7f27ac032e20 comp rx=0 tx=0).stop 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 shutdown_connections 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f27a00a2c10 0x7f27a00a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b8077ce0 0x7f27b81a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27b8077340 0x7f27b819e610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 --2- 172.21.15.111:0/1811060575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b8076100 0x7f27b819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 >> 172.21.15.111:0/1811060575 conn(0x7f27b80fef30 msgr2=0x7f27b8100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 shutdown_connections 2024-04-24T22:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:35.992+0000 7f27c0400640 1 -- 172.21.15.111:0/1811060575 wait complete. 2024-04-24T22:39:36.016 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-24T22:39:36.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T22:39:36.157 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: 2024-04-24T22:39:35.784+0000 7ffa1df22700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: 2024-04-24T22:39:35.784+0000 7ffa1df22700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:active 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: Cluster is now healthy 2024-04-24T22:39:36.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-24T22:39:36.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-24T22:39:36.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T22:39:36.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1811060575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.159 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:35 smithi171 ceph-mon[32337]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T22:39:36.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 -- 172.21.15.111:0/430903998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8101650 msgr2=0x7fbad8101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 --2- 172.21.15.111:0/430903998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8101650 0x7fbad8101a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009900 tx=0x7fbacc02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 -- 172.21.15.111:0/430903998 shutdown_connections 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 --2- 172.21.15.111:0/430903998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8103200 0x7fbad8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 --2- 172.21.15.111:0/430903998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8102840 0x7fbad8102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 --2- 172.21.15.111:0/430903998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8101650 0x7fbad8101a70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 -- 172.21.15.111:0/430903998 >> 172.21.15.111:0/430903998 conn(0x7fbad80fcbe0 msgr2=0x7fbad80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 -- 172.21.15.111:0/430903998 shutdown_connections 2024-04-24T22:39:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.226+0000 7fbade800640 1 -- 172.21.15.111:0/430903998 wait complete. 2024-04-24T22:39:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.227+0000 7fbade800640 1 Processor -- start 2024-04-24T22:39:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.227+0000 7fbade800640 1 -- start start 2024-04-24T22:39:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.227+0000 7fbade800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 0x7fbad8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbade800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbade800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 0x7fbad81a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbade800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d2f0 con 0x7fbad8101650 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbade800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d170 con 0x7fbad8102840 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbade800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d470 con 0x7fbad8103200 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 0x7fbad8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadde00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 0x7fbad81a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 0x7fbad8199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47706/0 (socket says 172.21.15.111:47706) 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadd400640 1 -- 172.21.15.111:0/2301139660 learned_addr learned my addr 172.21.15.111:0/2301139660 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:41024/0 (socket says 172.21.15.111:41024) 2024-04-24T22:39:36.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 -- 172.21.15.111:0/2301139660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 msgr2=0x7fbad81a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 0x7fbad81a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 -- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 msgr2=0x7fbad8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 0x7fbad8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadca00640 1 -- 172.21.15.111:0/2301139660 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad81a17f0 con 0x7fbad8102840 2024-04-24T22:39:36.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.228+0000 7fbadde00640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 0x7fbad81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbadca00640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9f0 tx=0x7fbac400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400ccc0 con 0x7fbad8102840 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4010910 con 0x7fbad8102840 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad81a1ae0 con 0x7fbad8102840 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4018800 con 0x7fbad8102840 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.229+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad81a2080 con 0x7fbad8102840 2024-04-24T22:39:36.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.230+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad81a2310 con 0x7fbad8102840 2024-04-24T22:39:36.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.234+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbac4010ac0 con 0x7fbad8102840 2024-04-24T22:39:36.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.235+0000 7fbad6000640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 0x7fbac00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.235+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbac40cf1b0 con 0x7fbad8102840 2024-04-24T22:39:36.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.235+0000 7fbadd400640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 0x7fbac00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.236+0000 7fbadd400640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 0x7fbac00a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbacc000c00 tx=0x7fbacc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.239+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbac409c1c0 con 0x7fbad8102840 2024-04-24T22:39:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T22:39:36.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:active 2024-04-24T22:39:36.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T22:39:36.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T22:39:36.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: Cluster is now healthy 2024-04-24T22:39:36.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-24T22:39:36.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-24T22:39:36.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T22:39:36.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1811060575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:35 smithi186 ceph-mon[32753]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T22:39:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/848904209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T22:39:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] up:active 2024-04-24T22:39:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T22:39:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: Cluster is now healthy 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1811060575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T22:39:36.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:35 smithi111 ceph-mon[27746]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T22:39:36.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.463+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbad8075090 con 0x7fbac00a2c10 2024-04-24T22:39:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.492+0000 7fbad6000640 1 -- 172.21.15.111:0/2301139660 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbad8075090 con 0x7fbac00a2c10 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.494+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 msgr2=0x7fbac00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.494+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 0x7fbac00a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbacc000c00 tx=0x7fbacc0315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.494+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 msgr2=0x7fbad819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.494+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9f0 tx=0x7fbac400edf0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.494+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 shutdown_connections 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbac00a2c10 0x7fbac00a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbad8103200 0x7fbad81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbad8102840 0x7fbad819a110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 --2- 172.21.15.111:0/2301139660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbad8101650 0x7fbad8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 >> 172.21.15.111:0/2301139660 conn(0x7fbad80fcbe0 msgr2=0x7fbad8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 shutdown_connections 2024-04-24T22:39:36.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.495+0000 7fbade800640 1 -- 172.21.15.111:0/2301139660 wait complete. 2024-04-24T22:39:36.515 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.710+0000 7f60b9800640 1 -- 172.21.15.111:0/3946893847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 msgr2=0x7f60b4105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 --2- 172.21.15.111:0/3946893847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b4105010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 -- 172.21.15.111:0/3946893847 shutdown_connections 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 --2- 172.21.15.111:0/3946893847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b4105550 0x7f60b410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 --2- 172.21.15.111:0/3946893847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b4105010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 --2- 172.21.15.111:0/3946893847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b41039a0 0x7f60b4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 -- 172.21.15.111:0/3946893847 >> 172.21.15.111:0/3946893847 conn(0x7f60b40fef30 msgr2=0x7f60b4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 -- 172.21.15.111:0/3946893847 shutdown_connections 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.711+0000 7f60b9800640 1 -- 172.21.15.111:0/3946893847 wait complete. 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 Processor -- start 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 -- start start 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b4105550 0x7f60b41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60b410f650 con 0x7f60b4105550 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f60b410f4d0 con 0x7f60b4104b90 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.712+0000 7f60b9800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f60b410f7d0 con 0x7f60b41039a0 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44888/0 (socket says 172.21.15.111:44888) 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 -- 172.21.15.111:0/1905218731 learned_addr learned my addr 172.21.15.111:0/1905218731 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b3400640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b4105550 0x7f60b41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 -- 172.21.15.111:0/1905218731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 msgr2=0x7f60b419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 -- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b4105550 msgr2=0x7f60b41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b4105550 0x7f60b41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 -- 172.21.15.111:0/1905218731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b41a1900 con 0x7f60b41039a0 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2000640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:36.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.713+0000 7f60b2a00640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d770 tx=0x7f60a400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.714+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4017070 con 0x7f60b41039a0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.714+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4009c30 con 0x7f60b41039a0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.714+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41a1bf0 con 0x7f60b41039a0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.714+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4003dd0 con 0x7f60b41039a0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.714+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41a2110 con 0x7f60b41039a0 2024-04-24T22:39:36.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.715+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b410acc0 con 0x7f60b41039a0 2024-04-24T22:39:36.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.717+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f60a4003f70 con 0x7f60b41039a0 2024-04-24T22:39:36.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.718+0000 7f60ab400640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 0x7f60980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.718+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f60a40ce8a0 con 0x7f60b41039a0 2024-04-24T22:39:36.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.718+0000 7f60b2000640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 0x7f60980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.719+0000 7f60b2000640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 0x7f60980a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f609c031440 tx=0x7f609c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.723+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f60a4009de0 con 0x7f60b41039a0 2024-04-24T22:39:36.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:39:36.473+0000 7f3450400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:39:36.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:39:36.473+0000 7f3450400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:39:36.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:39:36.473+0000 7f3450400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:39:36.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:39:36.473+0000 7f3450400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:39:36.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:39:36.473+0000 7f3450400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:39:36.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.951+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f60b4061c10 con 0x7f60980a2b90 2024-04-24T22:39:36.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.988+0000 7f60ab400640 1 -- 172.21.15.111:0/1905218731 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60b4061c10 con 0x7f60980a2b90 2024-04-24T22:39:36.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.990+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 msgr2=0x7f60980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.990+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 0x7f60980a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f609c031440 tx=0x7f609c0315b0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.990+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 msgr2=0x7f60b4199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.990+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d770 tx=0x7f60a400dc30 comp rx=0 tx=0).stop 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 shutdown_connections 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f60980a2b90 0x7f60980a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b4105550 0x7f60b41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b4104b90 0x7f60b419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 --2- 172.21.15.111:0/1905218731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b41039a0 0x7f60b4199cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 >> 172.21.15.111:0/1905218731 conn(0x7f60b40fef30 msgr2=0x7f60b4107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 shutdown_connections 2024-04-24T22:39:36.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:36.991+0000 7f60b9800640 1 -- 172.21.15.111:0/1905218731 wait complete. 2024-04-24T22:39:37.013 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T22:39:37.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 -- 172.21.15.111:0/283307161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 msgr2=0x7fb700101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 --2- 172.21.15.111:0/283307161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb700101bc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009860 tx=0x7fb6e802a340 comp rx=0 tx=0).stop 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 -- 172.21.15.111:0/283307161 shutdown_connections 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 --2- 172.21.15.111:0/283307161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 0x7fb700108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 --2- 172.21.15.111:0/283307161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb700101bc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 --2- 172.21.15.111:0/283307161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 0x7fb700100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 -- 172.21.15.111:0/283307161 >> 172.21.15.111:0/283307161 conn(0x7fb7000fbae0 msgr2=0x7fb7000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.222+0000 7fb705200640 1 -- 172.21.15.111:0/283307161 shutdown_connections 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.223+0000 7fb705200640 1 -- 172.21.15.111:0/283307161 wait complete. 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.223+0000 7fb705200640 1 Processor -- start 2024-04-24T22:39:37.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 -- start start 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 0x7fb700199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 0x7fb7001a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010c1e0 con 0x7fb700101740 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010c060 con 0x7fb700102100 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb705200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010c360 con 0x7fb700100550 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6fea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6fea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47750/0 (socket says 172.21.15.111:47750) 2024-04-24T22:39:37.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6fea00640 1 -- 172.21.15.111:0/2959936629 learned_addr learned my addr 172.21.15.111:0/2959936629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:37.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6ff400640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 0x7fb700199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6ffe00640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 0x7fb7001a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6fea00640 1 -- 172.21.15.111:0/2959936629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 msgr2=0x7fb700199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.224+0000 7fb6fea00640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 0x7fb700199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6fea00640 1 -- 172.21.15.111:0/2959936629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 msgr2=0x7fb7001a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6fea00640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 0x7fb7001a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6fea00640 1 -- 172.21.15.111:0/2959936629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7001a1780 con 0x7fb700101740 2024-04-24T22:39:37.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6fea00640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8035040 tx=0x7fb6e8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8037040 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8032770 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7001a1a10 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80046c0 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.225+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7001a1f50 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.226+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7001a2120 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.228+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb6e8004050 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.229+0000 7fb6f7e00640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 0x7fb6e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.230+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb6e80eee20 con 0x7fb700101740 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.230+0000 7fb6ff400640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 0x7fb6e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.230+0000 7fb6ff400640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 0x7fb6e40a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098d0 tx=0x7fb6f0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.234+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb6e80bbe30 con 0x7fb700101740 2024-04-24T22:39:37.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:36 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T22:39:37.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:36 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:active 2024-04-24T22:39:37.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:36 smithi186 ceph-mon[32753]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-24T22:39:37.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:36 smithi186 ceph-mon[32753]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:39:37.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:36 smithi186 ceph-mon[32753]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:37.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T22:39:37.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:active 2024-04-24T22:39:37.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-mon[27746]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-24T22:39:37.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-mon[27746]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:39:37.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:36 smithi111 ceph-mon[27746]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:37.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:36 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T22:39:37.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:36 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] up:active 2024-04-24T22:39:37.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:36 smithi171 ceph-mon[32337]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-24T22:39:37.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:36 smithi171 ceph-mon[32337]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:39:37.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:36 smithi171 ceph-mon[32337]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.446+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb7000fdce0 con 0x7fb6e40a2c10 2024-04-24T22:39:37.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.455+0000 7fb6f7e00640 1 -- 172.21.15.111:0/2959936629 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb7000fdce0 con 0x7fb6e40a2c10 2024-04-24T22:39:37.457 INFO:teuthology.orchestra.run.smithi111.stdout:/volumes/qa/sv_0/679f7bfa-4b71-4cfb-acd5-76c9b0610f19 2024-04-24T22:39:37.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 msgr2=0x7fb6e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 0x7fb6e40a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098d0 tx=0x7fb6f0010040 comp rx=0 tx=0).stop 2024-04-24T22:39:37.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 msgr2=0x7fb70019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8035040 tx=0x7fb6e8033d90 comp rx=0 tx=0).stop 2024-04-24T22:39:37.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 shutdown_connections 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb6e40a2c10 0x7fb6e40a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb700102100 0x7fb7001a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb700101740 0x7fb70019a0f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 --2- 172.21.15.111:0/2959936629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb700100550 0x7fb700199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 >> 172.21.15.111:0/2959936629 conn(0x7fb7000fbae0 msgr2=0x7fb7000fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 shutdown_connections 2024-04-24T22:39:37.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.458+0000 7fb705200640 1 -- 172.21.15.111:0/2959936629 wait complete. 2024-04-24T22:39:37.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T22:39:37.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 -- 172.21.15.111:0/193359475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 msgr2=0x7ff31c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 --2- 172.21.15.111:0/193359475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c104ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff308009900 tx=0x7ff30802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 -- 172.21.15.111:0/193359475 shutdown_connections 2024-04-24T22:39:37.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 --2- 172.21.15.111:0/193359475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 0x7ff31c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 --2- 172.21.15.111:0/193359475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c104ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 --2- 172.21.15.111:0/193359475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 0x7ff31c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 -- 172.21.15.111:0/193359475 >> 172.21.15.111:0/193359475 conn(0x7ff31c0fedf0 msgr2=0x7ff31c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 -- 172.21.15.111:0/193359475 shutdown_connections 2024-04-24T22:39:37.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.692+0000 7ff31b200640 1 -- 172.21.15.111:0/193359475 wait complete. 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 Processor -- start 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 -- start start 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 0x7ff31c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 0x7ff31c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10f550 con 0x7ff31c104a70 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10f3d0 con 0x7ff31c105430 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.693+0000 7ff31b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10f6d0 con 0x7ff31c103880 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47754/0 (socket says 172.21.15.111:47754) 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 -- 172.21.15.111:0/1827045272 learned_addr learned my addr 172.21.15.111:0/1827045272 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319e00640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 0x7ff31c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff31a800640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 0x7ff31c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 -- 172.21.15.111:0/1827045272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 msgr2=0x7ff31c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 0x7ff31c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 -- 172.21.15.111:0/1827045272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 msgr2=0x7ff31c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 0x7ff31c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 -- 172.21.15.111:0/1827045272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c1a17e0 con 0x7ff31c104a70 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff319400640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff308031440 tx=0x7ff308032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.694+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30803f030 con 0x7ff31c104a70 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.695+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308003ce0 con 0x7ff31c104a70 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.695+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308035070 con 0x7ff31c104a70 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.695+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1a1a70 con 0x7ff31c104a70 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.695+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1a1f10 con 0x7ff31c104a70 2024-04-24T22:39:37.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.696+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c10aba0 con 0x7ff31c104a70 2024-04-24T22:39:37.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.698+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff308042030 con 0x7ff31c104a70 2024-04-24T22:39:37.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.700+0000 7ff312a00640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 0x7ff3000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.700+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff3080ef3f0 con 0x7ff31c104a70 2024-04-24T22:39:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.700+0000 7ff319e00640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 0x7ff3000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.701+0000 7ff319e00640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 0x7ff3000a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff31c1048d0 tx=0x7ff30c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.704+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff3080bc400 con 0x7ff31c104a70 2024-04-24T22:39:37.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.933+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff31c061c10 con 0x7ff3000a2b90 2024-04-24T22:39:37.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.975+0000 7ff312a00640 1 -- 172.21.15.111:0/1827045272 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff31c061c10 con 0x7ff3000a2b90 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.977+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 msgr2=0x7ff3000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.977+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 0x7ff3000a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff31c1048d0 tx=0x7ff30c005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 msgr2=0x7ff31c19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff308031440 tx=0x7ff308032ce0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 shutdown_connections 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff3000a2b90 0x7ff3000a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff31c105430 0x7ff31c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff31c104a70 0x7ff31c19a100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 --2- 172.21.15.111:0/1827045272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff31c103880 0x7ff31c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 >> 172.21.15.111:0/1827045272 conn(0x7ff31c0fedf0 msgr2=0x7ff31c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 shutdown_connections 2024-04-24T22:39:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:37.978+0000 7ff31b200640 1 -- 172.21.15.111:0/1827045272 wait complete. 2024-04-24T22:39:38.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T22:39:38.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 -- 172.21.15.111:0/4062249065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 msgr2=0x7f3ad8076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 --2- 172.21.15.111:0/4062249065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad8076520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009860 tx=0x7f3ac802a340 comp rx=0 tx=0).stop 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 -- 172.21.15.111:0/4062249065 shutdown_connections 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 --2- 172.21.15.111:0/4062249065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 0x7f3ad8078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 --2- 172.21.15.111:0/4062249065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad80777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 --2- 172.21.15.111:0/4062249065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad8076520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 -- 172.21.15.111:0/4062249065 >> 172.21.15.111:0/4062249065 conn(0x7f3ad80fef30 msgr2=0x7f3ad8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 -- 172.21.15.111:0/4062249065 shutdown_connections 2024-04-24T22:39:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.200+0000 7f3ade000640 1 -- 172.21.15.111:0/4062249065 wait complete. 2024-04-24T22:39:38.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.201+0000 7f3ade000640 1 Processor -- start 2024-04-24T22:39:38.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 -- start start 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 0x7f3ad81a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113a30 con 0x7f3ad8077340 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81138b0 con 0x7f3ad8076100 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ade000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113bb0 con 0x7f3ad8077ce0 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47772/0 (socket says 172.21.15.111:47772) 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 -- 172.21.15.111:0/3234351579 learned_addr learned my addr 172.21.15.111:0/3234351579 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ad7400640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3ad7e00640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 0x7f3ad81a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 -- 172.21.15.111:0/3234351579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 msgr2=0x7f3ad81a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 0x7f3ad81a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 -- 172.21.15.111:0/3234351579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 msgr2=0x7f3ad819e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad819e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.202+0000 7f3acfe00640 1 -- 172.21.15.111:0/3234351579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad81a5d00 con 0x7f3ad8077340 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ad7400640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3acfe00640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000e9c0 tx=0x7f3ac000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000ccc0 con 0x7f3ad8077340 2024-04-24T22:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac0005500 con 0x7f3ad8077340 2024-04-24T22:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad81a5ff0 con 0x7f3ad8077340 2024-04-24T22:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0010430 con 0x7f3ad8077340 2024-04-24T22:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.203+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81a64b0 con 0x7f3ad8077340 2024-04-24T22:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.204+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad8110310 con 0x7f3ad8077340 2024-04-24T22:39:38.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.206+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3ac00046c0 con 0x7f3ad8077340 2024-04-24T22:39:38.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.207+0000 7f3ad4c00640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 0x7f3abc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.208+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3ac00cf550 con 0x7f3ad8077340 2024-04-24T22:39:38.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.208+0000 7f3ad7400640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 0x7f3abc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.208+0000 7f3ad7400640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 0x7f3abc0a50e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802a930 tx=0x7f3ac8030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.212+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ac009c400 con 0x7f3ad8077340 2024-04-24T22:39:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:37 smithi186 ceph-mon[32753]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:37 smithi186 ceph-mon[32753]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:37 smithi186 ceph-mon[32753]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-04-24T22:39:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:37 smithi111 ceph-mon[27746]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:37 smithi111 ceph-mon[27746]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:37 smithi111 ceph-mon[27746]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-04-24T22:39:38.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:37 smithi171 ceph-mon[32337]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:37 smithi171 ceph-mon[32337]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:38.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:37 smithi171 ceph-mon[32337]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-04-24T22:39:38.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.443+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3ad8061b80 con 0x7f3abc0a2c20 2024-04-24T22:39:38.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.454+0000 7f3ad4c00640 1 -- 172.21.15.111:0/3234351579 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3ad8061b80 con 0x7f3abc0a2c20 2024-04-24T22:39:38.457 INFO:teuthology.orchestra.run.smithi111.stdout:/volumes/qa/sv_1/3331d266-e75a-4000-8d95-5eee04103e7a 2024-04-24T22:39:38.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 msgr2=0x7f3abc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 0x7f3abc0a50e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802a930 tx=0x7f3ac8030550 comp rx=0 tx=0).stop 2024-04-24T22:39:38.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 msgr2=0x7f3ad819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000e9c0 tx=0x7f3ac000ee80 comp rx=0 tx=0).stop 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 shutdown_connections 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f3abc0a2c20 0x7f3abc0a50e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad8077ce0 0x7f3ad81a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ad8077340 0x7f3ad819e620 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 --2- 172.21.15.111:0/3234351579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ad8076100 0x7f3ad819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.457+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 >> 172.21.15.111:0/3234351579 conn(0x7f3ad80fef30 msgr2=0x7f3ad8100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.458+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 shutdown_connections 2024-04-24T22:39:38.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.458+0000 7f3ade000640 1 -- 172.21.15.111:0/3234351579 wait complete. 2024-04-24T22:39:38.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.689+0000 7f46f9400640 1 -- 172.21.15.111:0/709844009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 msgr2=0x7f46f40ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.689+0000 7f46f9400640 1 --2- 172.21.15.111:0/709844009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f40ff8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009900 tx=0x7f46dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 -- 172.21.15.111:0/709844009 shutdown_connections 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 --2- 172.21.15.111:0/709844009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f40ffe10 0x7f46f4101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 --2- 172.21.15.111:0/709844009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f40ff8d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 --2- 172.21.15.111:0/709844009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f4100980 0x7f46f4100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 -- 172.21.15.111:0/709844009 >> 172.21.15.111:0/709844009 conn(0x7f46f40faf30 msgr2=0x7f46f40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 -- 172.21.15.111:0/709844009 shutdown_connections 2024-04-24T22:39:38.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.690+0000 7f46f9400640 1 -- 172.21.15.111:0/709844009 wait complete. 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 Processor -- start 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 -- start start 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f40ffe10 0x7f46f419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f4100980 0x7f46f41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410f680 con 0x7f46f40ff450 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410f500 con 0x7f46f40ffe10 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.691+0000 7f46f9400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410f800 con 0x7f46f4100980 2024-04-24T22:39:38.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47788/0 (socket says 172.21.15.111:47788) 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 -- 172.21.15.111:0/801126815 learned_addr learned my addr 172.21.15.111:0/801126815 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f3200640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f4100980 0x7f46f41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 -- 172.21.15.111:0/801126815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f4100980 msgr2=0x7f46f41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f4100980 0x7f46f41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 -- 172.21.15.111:0/801126815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f40ffe10 msgr2=0x7f46f419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f40ffe10 0x7f46f419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 -- 172.21.15.111:0/801126815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f41a1900 con 0x7f46f40ff450 2024-04-24T22:39:38.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.692+0000 7f46f2800640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f46e400e9f0 tx=0x7f46e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.693+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e400cd80 con 0x7f46f40ff450 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.693+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e4010910 con 0x7f46f40ff450 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.693+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4018990 con 0x7f46f40ff450 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.693+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f41a1bf0 con 0x7f46f40ff450 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.693+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a20f0 con 0x7f46f40ff450 2024-04-24T22:39:38.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.694+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f410bb90 con 0x7f46f40ff450 2024-04-24T22:39:38.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.696+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f46e4010ac0 con 0x7f46f40ff450 2024-04-24T22:39:38.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.697+0000 7f46ebe00640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 0x7f46d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.698+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f46e40cf570 con 0x7f46f40ff450 2024-04-24T22:39:38.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.698+0000 7f46e9e00640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 0x7f46d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.698+0000 7f46e9e00640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 0x7f46d80a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0057c0 tx=0x7f46dc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.701+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46e409c420 con 0x7f46f40ff450 2024-04-24T22:39:38.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.942+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46f402d3d0 con 0x7f46f40ff450 2024-04-24T22:39:38.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.943+0000 7f46ebe00640 1 -- 172.21.15.111:0/801126815 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7f46e409bd20 con 0x7f46f40ff450 2024-04-24T22:39:38.946 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:38.946 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:39:38.946 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:39:38.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 msgr2=0x7f46d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 0x7f46d80a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0057c0 tx=0x7f46dc031400 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 msgr2=0x7f46f4199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f46e400e9f0 tx=0x7f46e400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 shutdown_connections 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46d80a2ca0 0x7f46d80a5160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46f4100980 0x7f46f41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f40ffe10 0x7f46f419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.946+0000 7f46f9400640 1 --2- 172.21.15.111:0/801126815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f40ff450 0x7f46f4199cf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.947+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 >> 172.21.15.111:0/801126815 conn(0x7f46f40faf30 msgr2=0x7f46f40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.947+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 shutdown_connections 2024-04-24T22:39:38.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:38.947+0000 7f46f9400640 1 -- 172.21.15.111:0/801126815 wait complete. 2024-04-24T22:39:38.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:39.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.168+0000 7f4516e00640 1 -- 172.21.15.111:0/1174165517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101037c0 msgr2=0x7f4510103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 --2- 172.21.15.111:0/1174165517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101037c0 0x7f4510103be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4500009ca0 tx=0x7f450002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 -- 172.21.15.111:0/1174165517 shutdown_connections 2024-04-24T22:39:39.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 --2- 172.21.15.111:0/1174165517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510105370 0x7f451010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 --2- 172.21.15.111:0/1174165517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45101049b0 0x7f4510104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 --2- 172.21.15.111:0/1174165517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101037c0 0x7f4510103be0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 -- 172.21.15.111:0/1174165517 >> 172.21.15.111:0/1174165517 conn(0x7f45100fed50 msgr2=0x7f4510101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 -- 172.21.15.111:0/1174165517 shutdown_connections 2024-04-24T22:39:39.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.169+0000 7f4516e00640 1 -- 172.21.15.111:0/1174165517 wait complete. 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 Processor -- start 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 -- start start 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45101037c0 0x7f4510199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4510105370 0x7f45101a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.170+0000 7f4516e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f451010f490 con 0x7f4510105370 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4516e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f451010f310 con 0x7f45101037c0 2024-04-24T22:39:39.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4516e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f451010f610 con 0x7f45101049b0 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44982/0 (socket says 172.21.15.111:44982) 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4516400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4510105370 0x7f45101a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 -- 172.21.15.111:0/1060525163 learned_addr learned my addr 172.21.15.111:0/1060525163 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515a00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45101037c0 0x7f4510199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 -- 172.21.15.111:0/1060525163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45101037c0 msgr2=0x7f4510199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45101037c0 0x7f4510199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 -- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4510105370 msgr2=0x7f45101a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4510105370 0x7f45101a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.171+0000 7f4515000640 1 -- 172.21.15.111:0/1060525163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45101a16f0 con 0x7f45101049b0 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f4515000640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00ea60 tx=0x7f44fc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00cde0 con 0x7f45101049b0 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc0035f0 con 0x7f45101049b0 2024-04-24T22:39:39.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45101a19e0 con 0x7f45101049b0 2024-04-24T22:39:39.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc010670 con 0x7f45101049b0 2024-04-24T22:39:39.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.172+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101a1f80 con 0x7f45101049b0 2024-04-24T22:39:39.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.173+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451010aae0 con 0x7f45101049b0 2024-04-24T22:39:39.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.176+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f44fc0037a0 con 0x7f45101049b0 2024-04-24T22:39:39.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.177+0000 7f450e000640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 0x7f44f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.178+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f44fc0cf390 con 0x7f45101049b0 2024-04-24T22:39:39.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.178+0000 7f4515a00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 0x7f44f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.179+0000 7f4515a00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 0x7f44f40a5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4500000f80 tx=0x7f4500005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.182+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f44fc09c240 con 0x7f45101049b0 2024-04-24T22:39:39.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:38 smithi111 ceph-mon[27746]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:38 smithi111 ceph-mon[27746]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T22:39:39.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:38 smithi111 ceph-mon[27746]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:38 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/801126815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:39.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.382+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f451002d3c0 con 0x7f45101049b0 2024-04-24T22:39:39.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.384+0000 7f450e000640 1 -- 172.21.15.111:0/1060525163 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7f44fc09bb40 con 0x7f45101049b0 2024-04-24T22:39:39.386 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:39.386 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":85,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:39:31.546250+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:39.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.386+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 msgr2=0x7f44f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 0x7f44f40a5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4500000f80 tx=0x7f4500005f30 comp rx=0 tx=0).stop 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 msgr2=0x7f451019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00ea60 tx=0x7f44fc00ef20 comp rx=0 tx=0).stop 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 shutdown_connections 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f44f40a2ca0 0x7f44f40a5160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4510105370 0x7f45101a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45101049b0 0x7f451019a010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 --2- 172.21.15.111:0/1060525163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45101037c0 0x7f4510199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 >> 172.21.15.111:0/1060525163 conn(0x7f45100fed50 msgr2=0x7f4510107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 shutdown_connections 2024-04-24T22:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.387+0000 7f4516e00640 1 -- 172.21.15.111:0/1060525163 wait complete. 2024-04-24T22:39:39.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:38 smithi171 ceph-mon[32337]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:38 smithi171 ceph-mon[32337]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T22:39:39.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:38 smithi171 ceph-mon[32337]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:38 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/801126815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:39.409 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T22:39:39.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:39.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:38 smithi186 ceph-mon[32753]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:38 smithi186 ceph-mon[32753]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T22:39:39.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:38 smithi186 ceph-mon[32753]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:39.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:38 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/801126815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 -- 172.21.15.111:0/914083326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec108db0 msgr2=0x7ff5ec109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/914083326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec108db0 0x7ff5ec109230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009900 tx=0x7ff5d402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 -- 172.21.15.111:0/914083326 shutdown_connections 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/914083326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 0x7ff5ec110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/914083326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec108db0 0x7ff5ec109230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/914083326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 -- 172.21.15.111:0/914083326 >> 172.21.15.111:0/914083326 conn(0x7ff5ec076b80 msgr2=0x7ff5ec078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 -- 172.21.15.111:0/914083326 shutdown_connections 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.608+0000 7ff5f0a00640 1 -- 172.21.15.111:0/914083326 wait complete. 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 Processor -- start 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 -- start start 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec108db0 0x7ff5ec19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 0x7ff5ec1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec113840 con 0x7ff5ec107bc0 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1136c0 con 0x7ff5ec109770 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.609+0000 7ff5f0a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1139c0 con 0x7ff5ec108db0 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47820/0 (socket says 172.21.15.111:47820) 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 -- 172.21.15.111:0/1120354382 learned_addr learned my addr 172.21.15.111:0/1120354382 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5e2a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec108db0 0x7ff5ec19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5ebe00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 0x7ff5ec1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 -- 172.21.15.111:0/1120354382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec108db0 msgr2=0x7ff5ec19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec108db0 0x7ff5ec19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 -- 172.21.15.111:0/1120354382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 msgr2=0x7ff5ec1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 0x7ff5ec1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 -- 172.21.15.111:0/1120354382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1a5ad0 con 0x7ff5ec107bc0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.610+0000 7ff5eb400640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9c0 tx=0x7ff5dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.611+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00cd80 con 0x7ff5ec107bc0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.611+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc0035f0 con 0x7ff5ec107bc0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.611+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc010690 con 0x7ff5ec107bc0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.611+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1a5dc0 con 0x7ff5ec107bc0 2024-04-24T22:39:39.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.611+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1a6170 con 0x7ff5ec107bc0 2024-04-24T22:39:39.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.612+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec10ef30 con 0x7ff5ec107bc0 2024-04-24T22:39:39.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.614+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff5dc0037a0 con 0x7ff5ec107bc0 2024-04-24T22:39:39.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.615+0000 7ff5e8c00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 0x7ff5d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.615+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff5dc0cf400 con 0x7ff5ec107bc0 2024-04-24T22:39:39.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.616+0000 7ff5e2a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 0x7ff5d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.616+0000 7ff5e2a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 0x7ff5d00a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40314e0 tx=0x7ff5d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.620+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff5dc09c2b0 con 0x7ff5ec107bc0 2024-04-24T22:39:39.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.817+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5ec061b80 con 0x7ff5d00a2ca0 2024-04-24T22:39:39.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.827+0000 7ff5e8c00640 1 -- 172.21.15.111:0/1120354382 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258705 (secure 0 0 0) 0x7ff5ec061b80 con 0x7ff5d00a2ca0 2024-04-24T22:39:39.833 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 msgr2=0x7ff5d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 0x7ff5d00a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40314e0 tx=0x7ff5d4000bc0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 msgr2=0x7ff5ec19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9c0 tx=0x7ff5dc00ee80 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 shutdown_connections 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff5d00a2ca0 0x7ff5d00a5160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.833+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5ec109770 0x7ff5ec1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.834+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5ec108db0 0x7ff5ec19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.834+0000 7ff5f0a00640 1 --2- 172.21.15.111:0/1120354382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ec107bc0 0x7ff5ec19deb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.834+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 >> 172.21.15.111:0/1120354382 conn(0x7ff5ec076b80 msgr2=0x7ff5ec10c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.834+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 shutdown_connections 2024-04-24T22:39:39.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:39.834+0000 7ff5f0a00640 1 -- 172.21.15.111:0/1120354382 wait complete. 2024-04-24T22:39:39.855 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-24T22:39:39.570477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595981,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":165,"num_read_kb":144,"num_write":306,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378512,"kb_used_data":12720,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563312,"statfs":{"total":1151940427776,"available":1151552831488,"internally_reserved":0,"allocated":13025280,"data_stored":7088096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":125,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.022838"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993005+0000","last_change":"2024-04-24T22:39:30.580615+0000","last_active":"2024-04-24T22:39:31.993005+0000","last_peered":"2024-04-24T22:39:31.993005+0000","last_clean":"2024-04-24T22:39:31.993005+0000","last_became_active":"2024-04-24T22:39:30.580299+0000","last_became_peered":"2024-04-24T22:39:30.580299+0000","last_unstale":"2024-04-24T22:39:31.993005+0000","last_undegraded":"2024-04-24T22:39:31.993005+0000","last_fullsized":"2024-04-24T22:39:31.993005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.364115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625609+0000","last_change":"2024-04-24T22:39:30.588495+0000","last_active":"2024-04-24T22:39:31.625609+0000","last_peered":"2024-04-24T22:39:31.625609+0000","last_clean":"2024-04-24T22:39:31.625609+0000","last_became_active":"2024-04-24T22:39:30.585588+0000","last_became_peered":"2024-04-24T22:39:30.585588+0000","last_unstale":"2024-04-24T22:39:31.625609+0000","last_undegraded":"2024-04-24T22:39:31.625609+0000","last_fullsized":"2024-04-24T22:39:31.625609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.894832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625461+0000","last_change":"2024-04-24T22:39:30.585139+0000","last_active":"2024-04-24T22:39:31.625461+0000","last_peered":"2024-04-24T22:39:31.625461+0000","last_clean":"2024-04-24T22:39:31.625461+0000","last_became_active":"2024-04-24T22:39:30.583299+0000","last_became_peered":"2024-04-24T22:39:30.583299+0000","last_unstale":"2024-04-24T22:39:31.625461+0000","last_undegraded":"2024-04-24T22:39:31.625461+0000","last_fullsized":"2024-04-24T22:39:31.625461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:20.941383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624460+0000","last_change":"2024-04-24T22:39:30.566540+0000","last_active":"2024-04-24T22:39:31.624460+0000","last_peered":"2024-04-24T22:39:31.624460+0000","last_clean":"2024-04-24T22:39:31.624460+0000","last_became_active":"2024-04-24T22:39:30.566195+0000","last_became_peered":"2024-04-24T22:39:30.566195+0000","last_unstale":"2024-04-24T22:39:31.624460+0000","last_undegraded":"2024-04-24T22:39:31.624460+0000","last_fullsized":"2024-04-24T22:39:31.624460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:24.538153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623321+0000","last_change":"2024-04-24T22:39:30.591842+0000","last_active":"2024-04-24T22:39:31.623321+0000","last_peered":"2024-04-24T22:39:31.623321+0000","last_clean":"2024-04-24T22:39:31.623321+0000","last_became_active":"2024-04-24T22:39:30.591579+0000","last_became_peered":"2024-04-24T22:39:30.591579+0000","last_unstale":"2024-04-24T22:39:31.623321+0000","last_undegraded":"2024-04-24T22:39:31.623321+0000","last_fullsized":"2024-04-24T22:39:31.623321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:48.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623682+0000","last_change":"2024-04-24T22:39:30.577689+0000","last_active":"2024-04-24T22:39:31.623682+0000","last_peered":"2024-04-24T22:39:31.623682+0000","last_clean":"2024-04-24T22:39:31.623682+0000","last_became_active":"2024-04-24T22:39:30.577362+0000","last_became_peered":"2024-04-24T22:39:30.577362+0000","last_unstale":"2024-04-24T22:39:31.623682+0000","last_undegraded":"2024-04-24T22:39:31.623682+0000","last_fullsized":"2024-04-24T22:39:31.623682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:52.808898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623234+0000","last_change":"2024-04-24T22:39:30.576400+0000","last_active":"2024-04-24T22:39:31.623234+0000","last_peered":"2024-04-24T22:39:31.623234+0000","last_clean":"2024-04-24T22:39:31.623234+0000","last_became_active":"2024-04-24T22:39:30.576141+0000","last_became_peered":"2024-04-24T22:39:30.576141+0000","last_unstale":"2024-04-24T22:39:31.623234+0000","last_undegraded":"2024-04-24T22:39:31.623234+0000","last_fullsized":"2024-04-24T22:39:31.623234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623755+0000","last_change":"2024-04-24T22:39:30.594496+0000","last_active":"2024-04-24T22:39:31.623755+0000","last_peered":"2024-04-24T22:39:31.623755+0000","last_clean":"2024-04-24T22:39:31.623755+0000","last_became_active":"2024-04-24T22:39:30.589541+0000","last_became_peered":"2024-04-24T22:39:30.589541+0000","last_unstale":"2024-04-24T22:39:31.623755+0000","last_undegraded":"2024-04-24T22:39:31.623755+0000","last_fullsized":"2024-04-24T22:39:31.623755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:50.593142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551116+0000","last_change":"2024-04-24T22:39:30.567922+0000","last_active":"2024-04-24T22:39:31.551116+0000","last_peered":"2024-04-24T22:39:31.551116+0000","last_clean":"2024-04-24T22:39:31.551116+0000","last_became_active":"2024-04-24T22:39:30.567470+0000","last_became_peered":"2024-04-24T22:39:30.567470+0000","last_unstale":"2024-04-24T22:39:31.551116+0000","last_undegraded":"2024-04-24T22:39:31.551116+0000","last_fullsized":"2024-04-24T22:39:31.551116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:11.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623972+0000","last_change":"2024-04-24T22:39:30.566716+0000","last_active":"2024-04-24T22:39:31.623972+0000","last_peered":"2024-04-24T22:39:31.623972+0000","last_clean":"2024-04-24T22:39:31.623972+0000","last_became_active":"2024-04-24T22:39:30.566192+0000","last_became_peered":"2024-04-24T22:39:30.566192+0000","last_unstale":"2024-04-24T22:39:31.623972+0000","last_undegraded":"2024-04-24T22:39:31.623972+0000","last_fullsized":"2024-04-24T22:39:31.623972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:36.364984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624739+0000","last_change":"2024-04-24T22:39:30.571679+0000","last_active":"2024-04-24T22:39:31.624739+0000","last_peered":"2024-04-24T22:39:31.624739+0000","last_clean":"2024-04-24T22:39:31.624739+0000","last_became_active":"2024-04-24T22:39:30.571375+0000","last_became_peered":"2024-04-24T22:39:30.571375+0000","last_unstale":"2024-04-24T22:39:31.624739+0000","last_undegraded":"2024-04-24T22:39:31.624739+0000","last_fullsized":"2024-04-24T22:39:31.624739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:47.219082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624257+0000","last_change":"2024-04-24T22:39:30.565158+0000","last_active":"2024-04-24T22:39:31.624257+0000","last_peered":"2024-04-24T22:39:31.624257+0000","last_clean":"2024-04-24T22:39:31.624257+0000","last_became_active":"2024-04-24T22:39:30.564808+0000","last_became_peered":"2024-04-24T22:39:30.564808+0000","last_unstale":"2024-04-24T22:39:31.624257+0000","last_undegraded":"2024-04-24T22:39:31.624257+0000","last_fullsized":"2024-04-24T22:39:31.624257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:33.534801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551139+0000","last_change":"2024-04-24T22:39:30.561125+0000","last_active":"2024-04-24T22:39:31.551139+0000","last_peered":"2024-04-24T22:39:31.551139+0000","last_clean":"2024-04-24T22:39:31.551139+0000","last_became_active":"2024-04-24T22:39:30.559473+0000","last_became_peered":"2024-04-24T22:39:30.559473+0000","last_unstale":"2024-04-24T22:39:31.551139+0000","last_undegraded":"2024-04-24T22:39:31.551139+0000","last_fullsized":"2024-04-24T22:39:31.551139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:09.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624224+0000","last_change":"2024-04-24T22:39:30.582371+0000","last_active":"2024-04-24T22:39:31.624224+0000","last_peered":"2024-04-24T22:39:31.624224+0000","last_clean":"2024-04-24T22:39:31.624224+0000","last_became_active":"2024-04-24T22:39:30.582002+0000","last_became_peered":"2024-04-24T22:39:30.582002+0000","last_unstale":"2024-04-24T22:39:31.624224+0000","last_undegraded":"2024-04-24T22:39:31.624224+0000","last_fullsized":"2024-04-24T22:39:31.624224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:11.812461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623952+0000","last_change":"2024-04-24T22:39:30.581249+0000","last_active":"2024-04-24T22:39:31.623952+0000","last_peered":"2024-04-24T22:39:31.623952+0000","last_clean":"2024-04-24T22:39:31.623952+0000","last_became_active":"2024-04-24T22:39:30.580820+0000","last_became_peered":"2024-04-24T22:39:30.580820+0000","last_unstale":"2024-04-24T22:39:31.623952+0000","last_undegraded":"2024-04-24T22:39:31.623952+0000","last_fullsized":"2024-04-24T22:39:31.623952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:58.940926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550624+0000","last_change":"2024-04-24T22:39:30.578310+0000","last_active":"2024-04-24T22:39:31.550624+0000","last_peered":"2024-04-24T22:39:31.550624+0000","last_clean":"2024-04-24T22:39:31.550624+0000","last_became_active":"2024-04-24T22:39:30.577917+0000","last_became_peered":"2024-04-24T22:39:30.577917+0000","last_unstale":"2024-04-24T22:39:31.550624+0000","last_undegraded":"2024-04-24T22:39:31.550624+0000","last_fullsized":"2024-04-24T22:39:31.550624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:51.766196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993166+0000","last_change":"2024-04-24T22:39:30.580939+0000","last_active":"2024-04-24T22:39:31.993166+0000","last_peered":"2024-04-24T22:39:31.993166+0000","last_clean":"2024-04-24T22:39:31.993166+0000","last_became_active":"2024-04-24T22:39:30.580583+0000","last_became_peered":"2024-04-24T22:39:30.580583+0000","last_unstale":"2024-04-24T22:39:31.993166+0000","last_undegraded":"2024-04-24T22:39:31.993166+0000","last_fullsized":"2024-04-24T22:39:31.993166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.891103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626593+0000","last_change":"2024-04-24T22:39:30.577933+0000","last_active":"2024-04-24T22:39:31.626593+0000","last_peered":"2024-04-24T22:39:31.626593+0000","last_clean":"2024-04-24T22:39:31.626593+0000","last_became_active":"2024-04-24T22:39:30.577545+0000","last_became_peered":"2024-04-24T22:39:30.577545+0000","last_unstale":"2024-04-24T22:39:31.626593+0000","last_undegraded":"2024-04-24T22:39:31.626593+0000","last_fullsized":"2024-04-24T22:39:31.626593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:26.876281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.553246+0000","last_change":"2024-04-24T22:39:30.586170+0000","last_active":"2024-04-24T22:39:31.553246+0000","last_peered":"2024-04-24T22:39:31.553246+0000","last_clean":"2024-04-24T22:39:31.553246+0000","last_became_active":"2024-04-24T22:39:30.585922+0000","last_became_peered":"2024-04-24T22:39:30.585922+0000","last_unstale":"2024-04-24T22:39:31.553246+0000","last_undegraded":"2024-04-24T22:39:31.553246+0000","last_fullsized":"2024-04-24T22:39:31.553246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:53.166725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.992381+0000","last_change":"2024-04-24T22:39:29.660103+0000","last_active":"2024-04-24T22:39:34.992381+0000","last_peered":"2024-04-24T22:39:34.992381+0000","last_clean":"2024-04-24T22:39:34.992381+0000","last_became_active":"2024-04-24T22:39:29.659809+0000","last_became_peered":"2024-04-24T22:39:29.659809+0000","last_unstale":"2024-04-24T22:39:34.992381+0000","last_undegraded":"2024-04-24T22:39:34.992381+0000","last_fullsized":"2024-04-24T22:39:34.992381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:40.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624143+0000","last_change":"2024-04-24T22:39:30.587041+0000","last_active":"2024-04-24T22:39:31.624143+0000","last_peered":"2024-04-24T22:39:31.624143+0000","last_clean":"2024-04-24T22:39:31.624143+0000","last_became_active":"2024-04-24T22:39:30.586568+0000","last_became_peered":"2024-04-24T22:39:30.586568+0000","last_unstale":"2024-04-24T22:39:31.624143+0000","last_undegraded":"2024-04-24T22:39:31.624143+0000","last_fullsized":"2024-04-24T22:39:31.624143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:13.305790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624532+0000","last_change":"2024-04-24T22:39:29.659591+0000","last_active":"2024-04-24T22:39:31.624532+0000","last_peered":"2024-04-24T22:39:31.624532+0000","last_clean":"2024-04-24T22:39:31.624532+0000","last_became_active":"2024-04-24T22:39:29.659281+0000","last_became_peered":"2024-04-24T22:39:29.659281+0000","last_unstale":"2024-04-24T22:39:31.624532+0000","last_undegraded":"2024-04-24T22:39:31.624532+0000","last_fullsized":"2024-04-24T22:39:31.624532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:07.909300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626309+0000","last_change":"2024-04-24T22:39:30.589041+0000","last_active":"2024-04-24T22:39:31.626309+0000","last_peered":"2024-04-24T22:39:31.626309+0000","last_clean":"2024-04-24T22:39:31.626309+0000","last_became_active":"2024-04-24T22:39:30.588706+0000","last_became_peered":"2024-04-24T22:39:30.588706+0000","last_unstale":"2024-04-24T22:39:31.626309+0000","last_undegraded":"2024-04-24T22:39:31.626309+0000","last_fullsized":"2024-04-24T22:39:31.626309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:19.594236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.986772+0000","last_change":"2024-04-24T22:39:29.659280+0000","last_active":"2024-04-24T22:39:34.986772+0000","last_peered":"2024-04-24T22:39:34.986772+0000","last_clean":"2024-04-24T22:39:34.986772+0000","last_became_active":"2024-04-24T22:39:29.658922+0000","last_became_peered":"2024-04-24T22:39:29.658922+0000","last_unstale":"2024-04-24T22:39:34.986772+0000","last_undegraded":"2024-04-24T22:39:34.986772+0000","last_fullsized":"2024-04-24T22:39:34.986772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:58.157675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550771+0000","last_change":"2024-04-24T22:39:30.573769+0000","last_active":"2024-04-24T22:39:31.550771+0000","last_peered":"2024-04-24T22:39:31.550771+0000","last_clean":"2024-04-24T22:39:31.550771+0000","last_became_active":"2024-04-24T22:39:30.573405+0000","last_became_peered":"2024-04-24T22:39:30.573405+0000","last_unstale":"2024-04-24T22:39:31.550771+0000","last_undegraded":"2024-04-24T22:39:31.550771+0000","last_fullsized":"2024-04-24T22:39:31.550771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:08.919808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623929+0000","last_change":"2024-04-24T22:39:29.653759+0000","last_active":"2024-04-24T22:39:31.623929+0000","last_peered":"2024-04-24T22:39:31.623929+0000","last_clean":"2024-04-24T22:39:31.623929+0000","last_became_active":"2024-04-24T22:39:29.653060+0000","last_became_peered":"2024-04-24T22:39:29.653060+0000","last_unstale":"2024-04-24T22:39:31.623929+0000","last_undegraded":"2024-04-24T22:39:31.623929+0000","last_fullsized":"2024-04-24T22:39:31.623929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:02.503014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623517+0000","last_change":"2024-04-24T22:39:30.563070+0000","last_active":"2024-04-24T22:39:31.623517+0000","last_peered":"2024-04-24T22:39:31.623517+0000","last_clean":"2024-04-24T22:39:31.623517+0000","last_became_active":"2024-04-24T22:39:30.562736+0000","last_became_peered":"2024-04-24T22:39:30.562736+0000","last_unstale":"2024-04-24T22:39:31.623517+0000","last_undegraded":"2024-04-24T22:39:31.623517+0000","last_fullsized":"2024-04-24T22:39:31.623517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:15.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000955+0000","last_change":"2024-04-24T22:39:29.657659+0000","last_active":"2024-04-24T22:39:36.000955+0000","last_peered":"2024-04-24T22:39:36.000955+0000","last_clean":"2024-04-24T22:39:36.000955+0000","last_became_active":"2024-04-24T22:39:29.657237+0000","last_became_peered":"2024-04-24T22:39:29.657237+0000","last_unstale":"2024-04-24T22:39:36.000955+0000","last_undegraded":"2024-04-24T22:39:36.000955+0000","last_fullsized":"2024-04-24T22:39:36.000955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:11.376959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623743+0000","last_change":"2024-04-24T22:39:30.577542+0000","last_active":"2024-04-24T22:39:31.623743+0000","last_peered":"2024-04-24T22:39:31.623743+0000","last_clean":"2024-04-24T22:39:31.623743+0000","last_became_active":"2024-04-24T22:39:30.577194+0000","last_became_peered":"2024-04-24T22:39:30.577194+0000","last_unstale":"2024-04-24T22:39:31.623743+0000","last_undegraded":"2024-04-24T22:39:31.623743+0000","last_fullsized":"2024-04-24T22:39:31.623743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:26.000787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991451+0000","last_change":"2024-04-24T22:39:29.671709+0000","last_active":"2024-04-24T22:39:34.991451+0000","last_peered":"2024-04-24T22:39:34.991451+0000","last_clean":"2024-04-24T22:39:34.991451+0000","last_became_active":"2024-04-24T22:39:29.671306+0000","last_became_peered":"2024-04-24T22:39:29.671306+0000","last_unstale":"2024-04-24T22:39:34.991451+0000","last_undegraded":"2024-04-24T22:39:34.991451+0000","last_fullsized":"2024-04-24T22:39:34.991451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:33.833217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623268+0000","last_change":"2024-04-24T22:39:30.583720+0000","last_active":"2024-04-24T22:39:31.623268+0000","last_peered":"2024-04-24T22:39:31.623268+0000","last_clean":"2024-04-24T22:39:31.623268+0000","last_became_active":"2024-04-24T22:39:30.583468+0000","last_became_peered":"2024-04-24T22:39:30.583468+0000","last_unstale":"2024-04-24T22:39:31.623268+0000","last_undegraded":"2024-04-24T22:39:31.623268+0000","last_fullsized":"2024-04-24T22:39:31.623268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:07.679284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.992195+0000","last_change":"2024-04-24T22:39:29.669100+0000","last_active":"2024-04-24T22:39:34.992195+0000","last_peered":"2024-04-24T22:39:34.992195+0000","last_clean":"2024-04-24T22:39:34.992195+0000","last_became_active":"2024-04-24T22:39:29.668773+0000","last_became_peered":"2024-04-24T22:39:29.668773+0000","last_unstale":"2024-04-24T22:39:34.992195+0000","last_undegraded":"2024-04-24T22:39:34.992195+0000","last_fullsized":"2024-04-24T22:39:34.992195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:56.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624052+0000","last_change":"2024-04-24T22:39:30.594562+0000","last_active":"2024-04-24T22:39:31.624052+0000","last_peered":"2024-04-24T22:39:31.624052+0000","last_clean":"2024-04-24T22:39:31.624052+0000","last_became_active":"2024-04-24T22:39:30.589928+0000","last_became_peered":"2024-04-24T22:39:30.589928+0000","last_unstale":"2024-04-24T22:39:31.624052+0000","last_undegraded":"2024-04-24T22:39:31.624052+0000","last_fullsized":"2024-04-24T22:39:31.624052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:14.951690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623174+0000","last_change":"2024-04-24T22:39:29.658615+0000","last_active":"2024-04-24T22:39:31.623174+0000","last_peered":"2024-04-24T22:39:31.623174+0000","last_clean":"2024-04-24T22:39:31.623174+0000","last_became_active":"2024-04-24T22:39:29.658178+0000","last_became_peered":"2024-04-24T22:39:29.658178+0000","last_unstale":"2024-04-24T22:39:31.623174+0000","last_undegraded":"2024-04-24T22:39:31.623174+0000","last_fullsized":"2024-04-24T22:39:31.623174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.821509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625744+0000","last_change":"2024-04-24T22:39:30.592142+0000","last_active":"2024-04-24T22:39:31.625744+0000","last_peered":"2024-04-24T22:39:31.625744+0000","last_clean":"2024-04-24T22:39:31.625744+0000","last_became_active":"2024-04-24T22:39:30.591417+0000","last_became_peered":"2024-04-24T22:39:30.591417+0000","last_unstale":"2024-04-24T22:39:31.625744+0000","last_undegraded":"2024-04-24T22:39:31.625744+0000","last_fullsized":"2024-04-24T22:39:31.625744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:55.818795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.978374+0000","last_change":"2024-04-24T22:39:29.662513+0000","last_active":"2024-04-24T22:39:33.978374+0000","last_peered":"2024-04-24T22:39:33.978374+0000","last_clean":"2024-04-24T22:39:33.978374+0000","last_became_active":"2024-04-24T22:39:29.662223+0000","last_became_peered":"2024-04-24T22:39:29.662223+0000","last_unstale":"2024-04-24T22:39:33.978374+0000","last_undegraded":"2024-04-24T22:39:33.978374+0000","last_fullsized":"2024-04-24T22:39:33.978374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.012595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550956+0000","last_change":"2024-04-24T22:39:30.577594+0000","last_active":"2024-04-24T22:39:31.550956+0000","last_peered":"2024-04-24T22:39:31.550956+0000","last_clean":"2024-04-24T22:39:31.550956+0000","last_became_active":"2024-04-24T22:39:30.577254+0000","last_became_peered":"2024-04-24T22:39:30.577254+0000","last_unstale":"2024-04-24T22:39:31.550956+0000","last_undegraded":"2024-04-24T22:39:31.550956+0000","last_fullsized":"2024-04-24T22:39:31.550956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:47.926878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.973884+0000","last_change":"2024-04-24T22:39:29.659898+0000","last_active":"2024-04-24T22:39:33.973884+0000","last_peered":"2024-04-24T22:39:33.973884+0000","last_clean":"2024-04-24T22:39:33.973884+0000","last_became_active":"2024-04-24T22:39:29.659582+0000","last_became_peered":"2024-04-24T22:39:29.659582+0000","last_unstale":"2024-04-24T22:39:33.973884+0000","last_undegraded":"2024-04-24T22:39:33.973884+0000","last_fullsized":"2024-04-24T22:39:33.973884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626164+0000","last_change":"2024-04-24T22:39:30.588271+0000","last_active":"2024-04-24T22:39:31.626164+0000","last_peered":"2024-04-24T22:39:31.626164+0000","last_clean":"2024-04-24T22:39:31.626164+0000","last_became_active":"2024-04-24T22:39:30.578846+0000","last_became_peered":"2024-04-24T22:39:30.578846+0000","last_unstale":"2024-04-24T22:39:31.626164+0000","last_undegraded":"2024-04-24T22:39:31.626164+0000","last_fullsized":"2024-04-24T22:39:31.626164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:23.807550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.976436+0000","last_change":"2024-04-24T22:39:29.654708+0000","last_active":"2024-04-24T22:39:33.976436+0000","last_peered":"2024-04-24T22:39:33.976436+0000","last_clean":"2024-04-24T22:39:33.976436+0000","last_became_active":"2024-04-24T22:39:29.654290+0000","last_became_peered":"2024-04-24T22:39:29.654290+0000","last_unstale":"2024-04-24T22:39:33.976436+0000","last_undegraded":"2024-04-24T22:39:33.976436+0000","last_fullsized":"2024-04-24T22:39:33.976436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:29.445700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550820+0000","last_change":"2024-04-24T22:39:30.569436+0000","last_active":"2024-04-24T22:39:31.550820+0000","last_peered":"2024-04-24T22:39:31.550820+0000","last_clean":"2024-04-24T22:39:31.550820+0000","last_became_active":"2024-04-24T22:39:30.569104+0000","last_became_peered":"2024-04-24T22:39:30.569104+0000","last_unstale":"2024-04-24T22:39:31.550820+0000","last_undegraded":"2024-04-24T22:39:31.550820+0000","last_fullsized":"2024-04-24T22:39:31.550820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:30.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000867+0000","last_change":"2024-04-24T22:39:29.657883+0000","last_active":"2024-04-24T22:39:36.000867+0000","last_peered":"2024-04-24T22:39:36.000867+0000","last_clean":"2024-04-24T22:39:36.000867+0000","last_became_active":"2024-04-24T22:39:29.657496+0000","last_became_peered":"2024-04-24T22:39:29.657496+0000","last_unstale":"2024-04-24T22:39:36.000867+0000","last_undegraded":"2024-04-24T22:39:36.000867+0000","last_fullsized":"2024-04-24T22:39:36.000867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:51:32.037937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624173+0000","last_change":"2024-04-24T22:39:30.569744+0000","last_active":"2024-04-24T22:39:31.624173+0000","last_peered":"2024-04-24T22:39:31.624173+0000","last_clean":"2024-04-24T22:39:31.624173+0000","last_became_active":"2024-04-24T22:39:30.569392+0000","last_became_peered":"2024-04-24T22:39:30.569392+0000","last_unstale":"2024-04-24T22:39:31.624173+0000","last_undegraded":"2024-04-24T22:39:31.624173+0000","last_fullsized":"2024-04-24T22:39:31.624173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:09.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.993575+0000","last_change":"2024-04-24T22:39:29.657646+0000","last_active":"2024-04-24T22:39:34.993575+0000","last_peered":"2024-04-24T22:39:34.993575+0000","last_clean":"2024-04-24T22:39:34.993575+0000","last_became_active":"2024-04-24T22:39:29.657302+0000","last_became_peered":"2024-04-24T22:39:29.657302+0000","last_unstale":"2024-04-24T22:39:34.993575+0000","last_undegraded":"2024-04-24T22:39:34.993575+0000","last_fullsized":"2024-04-24T22:39:34.993575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:01.449788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551026+0000","last_change":"2024-04-24T22:39:30.591911+0000","last_active":"2024-04-24T22:39:31.551026+0000","last_peered":"2024-04-24T22:39:31.551026+0000","last_clean":"2024-04-24T22:39:31.551026+0000","last_became_active":"2024-04-24T22:39:30.591520+0000","last_became_peered":"2024-04-24T22:39:30.591520+0000","last_unstale":"2024-04-24T22:39:31.551026+0000","last_undegraded":"2024-04-24T22:39:31.551026+0000","last_fullsized":"2024-04-24T22:39:31.551026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:02.545529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975200+0000","last_change":"2024-04-24T22:39:29.657561+0000","last_active":"2024-04-24T22:39:33.975200+0000","last_peered":"2024-04-24T22:39:33.975200+0000","last_clean":"2024-04-24T22:39:33.975200+0000","last_became_active":"2024-04-24T22:39:29.657261+0000","last_became_peered":"2024-04-24T22:39:29.657261+0000","last_unstale":"2024-04-24T22:39:33.975200+0000","last_undegraded":"2024-04-24T22:39:33.975200+0000","last_fullsized":"2024-04-24T22:39:33.975200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:11.856640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624016+0000","last_change":"2024-04-24T22:39:30.577405+0000","last_active":"2024-04-24T22:39:31.624016+0000","last_peered":"2024-04-24T22:39:31.624016+0000","last_clean":"2024-04-24T22:39:31.624016+0000","last_became_active":"2024-04-24T22:39:30.576980+0000","last_became_peered":"2024-04-24T22:39:30.576980+0000","last_unstale":"2024-04-24T22:39:31.624016+0000","last_undegraded":"2024-04-24T22:39:31.624016+0000","last_fullsized":"2024-04-24T22:39:31.624016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:00.145497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:37.005193+0000","last_change":"2024-04-24T22:39:29.661054+0000","last_active":"2024-04-24T22:39:37.005193+0000","last_peered":"2024-04-24T22:39:37.005193+0000","last_clean":"2024-04-24T22:39:37.005193+0000","last_became_active":"2024-04-24T22:39:29.660638+0000","last_became_peered":"2024-04-24T22:39:29.660638+0000","last_unstale":"2024-04-24T22:39:37.005193+0000","last_undegraded":"2024-04-24T22:39:37.005193+0000","last_fullsized":"2024-04-24T22:39:37.005193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:51.891654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623158+0000","last_change":"2024-04-24T22:39:30.570407+0000","last_active":"2024-04-24T22:39:31.623158+0000","last_peered":"2024-04-24T22:39:31.623158+0000","last_clean":"2024-04-24T22:39:31.623158+0000","last_became_active":"2024-04-24T22:39:30.570070+0000","last_became_peered":"2024-04-24T22:39:30.570070+0000","last_unstale":"2024-04-24T22:39:31.623158+0000","last_undegraded":"2024-04-24T22:39:31.623158+0000","last_fullsized":"2024-04-24T22:39:31.623158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991910+0000","last_change":"2024-04-24T22:39:29.667020+0000","last_active":"2024-04-24T22:39:34.991910+0000","last_peered":"2024-04-24T22:39:34.991910+0000","last_clean":"2024-04-24T22:39:34.991910+0000","last_became_active":"2024-04-24T22:39:29.666427+0000","last_became_peered":"2024-04-24T22:39:29.666427+0000","last_unstale":"2024-04-24T22:39:34.991910+0000","last_undegraded":"2024-04-24T22:39:34.991910+0000","last_fullsized":"2024-04-24T22:39:34.991910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:49.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623048+0000","last_change":"2024-04-24T22:39:30.562092+0000","last_active":"2024-04-24T22:39:31.623048+0000","last_peered":"2024-04-24T22:39:31.623048+0000","last_clean":"2024-04-24T22:39:31.623048+0000","last_became_active":"2024-04-24T22:39:30.561704+0000","last_became_peered":"2024-04-24T22:39:30.561704+0000","last_unstale":"2024-04-24T22:39:31.623048+0000","last_undegraded":"2024-04-24T22:39:31.623048+0000","last_fullsized":"2024-04-24T22:39:31.623048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.996837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624054+0000","last_change":"2024-04-24T22:39:29.655612+0000","last_active":"2024-04-24T22:39:31.624054+0000","last_peered":"2024-04-24T22:39:31.624054+0000","last_clean":"2024-04-24T22:39:31.624054+0000","last_became_active":"2024-04-24T22:39:29.655212+0000","last_became_peered":"2024-04-24T22:39:29.655212+0000","last_unstale":"2024-04-24T22:39:31.624054+0000","last_undegraded":"2024-04-24T22:39:31.624054+0000","last_fullsized":"2024-04-24T22:39:31.624054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.414774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993019+0000","last_change":"2024-04-24T22:39:30.571705+0000","last_active":"2024-04-24T22:39:31.993019+0000","last_peered":"2024-04-24T22:39:31.993019+0000","last_clean":"2024-04-24T22:39:31.993019+0000","last_became_active":"2024-04-24T22:39:30.571158+0000","last_became_peered":"2024-04-24T22:39:30.571158+0000","last_unstale":"2024-04-24T22:39:31.993019+0000","last_undegraded":"2024-04-24T22:39:31.993019+0000","last_fullsized":"2024-04-24T22:39:31.993019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:45:47.935928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626544+0000","last_change":"2024-04-24T22:39:29.661052+0000","last_active":"2024-04-24T22:39:31.626544+0000","last_peered":"2024-04-24T22:39:31.626544+0000","last_clean":"2024-04-24T22:39:31.626544+0000","last_became_active":"2024-04-24T22:39:29.660716+0000","last_became_peered":"2024-04-24T22:39:29.660716+0000","last_unstale":"2024-04-24T22:39:31.626544+0000","last_undegraded":"2024-04-24T22:39:31.626544+0000","last_fullsized":"2024-04-24T22:39:31.626544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:52.016503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624855+0000","last_change":"2024-04-24T22:39:30.585190+0000","last_active":"2024-04-24T22:39:31.624855+0000","last_peered":"2024-04-24T22:39:31.624855+0000","last_clean":"2024-04-24T22:39:31.624855+0000","last_became_active":"2024-04-24T22:39:30.584846+0000","last_became_peered":"2024-04-24T22:39:30.584846+0000","last_unstale":"2024-04-24T22:39:31.624855+0000","last_undegraded":"2024-04-24T22:39:31.624855+0000","last_fullsized":"2024-04-24T22:39:31.624855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:07.094297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.795109+0000","last_change":"2024-04-24T22:39:29.671686+0000","last_active":"2024-04-24T22:39:31.795109+0000","last_peered":"2024-04-24T22:39:31.795109+0000","last_clean":"2024-04-24T22:39:31.795109+0000","last_became_active":"2024-04-24T22:39:29.671338+0000","last_became_peered":"2024-04-24T22:39:29.671338+0000","last_unstale":"2024-04-24T22:39:31.795109+0000","last_undegraded":"2024-04-24T22:39:31.795109+0000","last_fullsized":"2024-04-24T22:39:31.795109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:13.709429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623523+0000","last_change":"2024-04-24T22:39:30.585560+0000","last_active":"2024-04-24T22:39:31.623523+0000","last_peered":"2024-04-24T22:39:31.623523+0000","last_clean":"2024-04-24T22:39:31.623523+0000","last_became_active":"2024-04-24T22:39:30.585251+0000","last_became_peered":"2024-04-24T22:39:30.585251+0000","last_unstale":"2024-04-24T22:39:31.623523+0000","last_undegraded":"2024-04-24T22:39:31.623523+0000","last_fullsized":"2024-04-24T22:39:31.623523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:29.680556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623315+0000","last_change":"2024-04-24T22:39:30.569105+0000","last_active":"2024-04-24T22:39:31.623315+0000","last_peered":"2024-04-24T22:39:31.623315+0000","last_clean":"2024-04-24T22:39:31.623315+0000","last_became_active":"2024-04-24T22:39:30.566878+0000","last_became_peered":"2024-04-24T22:39:30.566878+0000","last_unstale":"2024-04-24T22:39:31.623315+0000","last_undegraded":"2024-04-24T22:39:31.623315+0000","last_fullsized":"2024-04-24T22:39:31.623315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.334294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.002064+0000","last_change":"2024-04-24T22:39:29.674334+0000","last_active":"2024-04-24T22:39:36.002064+0000","last_peered":"2024-04-24T22:39:36.002064+0000","last_clean":"2024-04-24T22:39:36.002064+0000","last_became_active":"2024-04-24T22:39:29.673968+0000","last_became_peered":"2024-04-24T22:39:29.673968+0000","last_unstale":"2024-04-24T22:39:36.002064+0000","last_undegraded":"2024-04-24T22:39:36.002064+0000","last_fullsized":"2024-04-24T22:39:36.002064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:10.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624791+0000","last_change":"2024-04-24T22:39:30.579663+0000","last_active":"2024-04-24T22:39:31.624791+0000","last_peered":"2024-04-24T22:39:31.624791+0000","last_clean":"2024-04-24T22:39:31.624791+0000","last_became_active":"2024-04-24T22:39:30.578730+0000","last_became_peered":"2024-04-24T22:39:30.578730+0000","last_unstale":"2024-04-24T22:39:31.624791+0000","last_undegraded":"2024-04-24T22:39:31.624791+0000","last_fullsized":"2024-04-24T22:39:31.624791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:25.604081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000002+0000","last_change":"2024-04-24T22:39:29.655316+0000","last_active":"2024-04-24T22:39:36.000002+0000","last_peered":"2024-04-24T22:39:36.000002+0000","last_clean":"2024-04-24T22:39:36.000002+0000","last_became_active":"2024-04-24T22:39:29.654975+0000","last_became_peered":"2024-04-24T22:39:29.654975+0000","last_unstale":"2024-04-24T22:39:36.000002+0000","last_undegraded":"2024-04-24T22:39:36.000002+0000","last_fullsized":"2024-04-24T22:39:36.000002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:20.461744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623223+0000","last_change":"2024-04-24T22:38:09.915368+0000","last_active":"2024-04-24T22:39:31.623223+0000","last_peered":"2024-04-24T22:39:31.623223+0000","last_clean":"2024-04-24T22:39:31.623223+0000","last_became_active":"2024-04-24T22:38:09.914879+0000","last_became_peered":"2024-04-24T22:38:09.914879+0000","last_unstale":"2024-04-24T22:39:31.623223+0000","last_undegraded":"2024-04-24T22:39:31.623223+0000","last_fullsized":"2024-04-24T22:39:31.623223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_clean_scrub_stamp":"2024-04-24T22:34:20.136986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:31.054930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550576+0000","last_change":"2024-04-24T22:39:30.568169+0000","last_active":"2024-04-24T22:39:31.550576+0000","last_peered":"2024-04-24T22:39:31.550576+0000","last_clean":"2024-04-24T22:39:31.550576+0000","last_became_active":"2024-04-24T22:39:30.567221+0000","last_became_peered":"2024-04-24T22:39:30.567221+0000","last_unstale":"2024-04-24T22:39:31.550576+0000","last_undegraded":"2024-04-24T22:39:31.550576+0000","last_fullsized":"2024-04-24T22:39:31.550576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:06.414726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.984607+0000","last_change":"2024-04-24T22:39:29.660378+0000","last_active":"2024-04-24T22:39:34.984607+0000","last_peered":"2024-04-24T22:39:34.984607+0000","last_clean":"2024-04-24T22:39:34.984607+0000","last_became_active":"2024-04-24T22:39:29.660050+0000","last_became_peered":"2024-04-24T22:39:29.660050+0000","last_unstale":"2024-04-24T22:39:34.984607+0000","last_undegraded":"2024-04-24T22:39:34.984607+0000","last_fullsized":"2024-04-24T22:39:34.984607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:06.199365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.977529+0000","last_change":"2024-04-24T22:39:29.674925+0000","last_active":"2024-04-24T22:39:33.977529+0000","last_peered":"2024-04-24T22:39:33.977529+0000","last_clean":"2024-04-24T22:39:33.977529+0000","last_became_active":"2024-04-24T22:39:29.674681+0000","last_became_peered":"2024-04-24T22:39:29.674681+0000","last_unstale":"2024-04-24T22:39:33.977529+0000","last_undegraded":"2024-04-24T22:39:33.977529+0000","last_fullsized":"2024-04-24T22:39:33.977529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:04.295567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551107+0000","last_change":"2024-04-24T22:39:30.581061+0000","last_active":"2024-04-24T22:39:31.551107+0000","last_peered":"2024-04-24T22:39:31.551107+0000","last_clean":"2024-04-24T22:39:31.551107+0000","last_became_active":"2024-04-24T22:39:30.579807+0000","last_became_peered":"2024-04-24T22:39:30.579807+0000","last_unstale":"2024-04-24T22:39:31.551107+0000","last_undegraded":"2024-04-24T22:39:31.551107+0000","last_fullsized":"2024-04-24T22:39:31.551107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:18.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.993660+0000","last_change":"2024-04-24T22:39:29.659948+0000","last_active":"2024-04-24T22:39:34.993660+0000","last_peered":"2024-04-24T22:39:34.993660+0000","last_clean":"2024-04-24T22:39:34.993660+0000","last_became_active":"2024-04-24T22:39:29.659606+0000","last_became_peered":"2024-04-24T22:39:29.659606+0000","last_unstale":"2024-04-24T22:39:34.993660+0000","last_undegraded":"2024-04-24T22:39:34.993660+0000","last_fullsized":"2024-04-24T22:39:34.993660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:32.877670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625120+0000","last_change":"2024-04-24T22:39:30.587740+0000","last_active":"2024-04-24T22:39:31.625120+0000","last_peered":"2024-04-24T22:39:31.625120+0000","last_clean":"2024-04-24T22:39:31.625120+0000","last_became_active":"2024-04-24T22:39:30.587063+0000","last_became_peered":"2024-04-24T22:39:30.587063+0000","last_unstale":"2024-04-24T22:39:31.625120+0000","last_undegraded":"2024-04-24T22:39:31.625120+0000","last_fullsized":"2024-04-24T22:39:31.625120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:02.704753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975036+0000","last_change":"2024-04-24T22:39:29.672345+0000","last_active":"2024-04-24T22:39:33.975036+0000","last_peered":"2024-04-24T22:39:33.975036+0000","last_clean":"2024-04-24T22:39:33.975036+0000","last_became_active":"2024-04-24T22:39:29.671957+0000","last_became_peered":"2024-04-24T22:39:29.671957+0000","last_unstale":"2024-04-24T22:39:33.975036+0000","last_undegraded":"2024-04-24T22:39:33.975036+0000","last_fullsized":"2024-04-24T22:39:33.975036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.181431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625761+0000","last_change":"2024-04-24T22:39:30.595273+0000","last_active":"2024-04-24T22:39:31.625761+0000","last_peered":"2024-04-24T22:39:31.625761+0000","last_clean":"2024-04-24T22:39:31.625761+0000","last_became_active":"2024-04-24T22:39:30.594765+0000","last_became_peered":"2024-04-24T22:39:30.594765+0000","last_unstale":"2024-04-24T22:39:31.625761+0000","last_undegraded":"2024-04-24T22:39:31.625761+0000","last_fullsized":"2024-04-24T22:39:31.625761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.836241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975351+0000","last_change":"2024-04-24T22:39:29.663779+0000","last_active":"2024-04-24T22:39:33.975351+0000","last_peered":"2024-04-24T22:39:33.975351+0000","last_clean":"2024-04-24T22:39:33.975351+0000","last_became_active":"2024-04-24T22:39:29.663484+0000","last_became_peered":"2024-04-24T22:39:29.663484+0000","last_unstale":"2024-04-24T22:39:33.975351+0000","last_undegraded":"2024-04-24T22:39:33.975351+0000","last_fullsized":"2024-04-24T22:39:33.975351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:37.322047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623365+0000","last_change":"2024-04-24T22:39:30.568608+0000","last_active":"2024-04-24T22:39:31.623365+0000","last_peered":"2024-04-24T22:39:31.623365+0000","last_clean":"2024-04-24T22:39:31.623365+0000","last_became_active":"2024-04-24T22:39:30.568271+0000","last_became_peered":"2024-04-24T22:39:30.568271+0000","last_unstale":"2024-04-24T22:39:31.623365+0000","last_undegraded":"2024-04-24T22:39:31.623365+0000","last_fullsized":"2024-04-24T22:39:31.623365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:31.433923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.977036+0000","last_change":"2024-04-24T22:39:29.663745+0000","last_active":"2024-04-24T22:39:33.977036+0000","last_peered":"2024-04-24T22:39:33.977036+0000","last_clean":"2024-04-24T22:39:33.977036+0000","last_became_active":"2024-04-24T22:39:29.663436+0000","last_became_peered":"2024-04-24T22:39:29.663436+0000","last_unstale":"2024-04-24T22:39:33.977036+0000","last_undegraded":"2024-04-24T22:39:33.977036+0000","last_fullsized":"2024-04-24T22:39:33.977036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.502177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992744+0000","last_change":"2024-04-24T22:39:30.580645+0000","last_active":"2024-04-24T22:39:31.992744+0000","last_peered":"2024-04-24T22:39:31.992744+0000","last_clean":"2024-04-24T22:39:31.992744+0000","last_became_active":"2024-04-24T22:39:30.580288+0000","last_became_peered":"2024-04-24T22:39:30.580288+0000","last_unstale":"2024-04-24T22:39:31.992744+0000","last_undegraded":"2024-04-24T22:39:31.992744+0000","last_fullsized":"2024-04-24T22:39:31.992744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:14.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.644717+0000","last_change":"2024-04-24T22:39:29.657571+0000","last_active":"2024-04-24T22:39:33.644717+0000","last_peered":"2024-04-24T22:39:33.644717+0000","last_clean":"2024-04-24T22:39:33.644717+0000","last_became_active":"2024-04-24T22:39:29.657230+0000","last_became_peered":"2024-04-24T22:39:29.657230+0000","last_unstale":"2024-04-24T22:39:33.644717+0000","last_undegraded":"2024-04-24T22:39:33.644717+0000","last_fullsized":"2024-04-24T22:39:33.644717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:26.638528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625230+0000","last_change":"2024-04-24T22:39:30.590468+0000","last_active":"2024-04-24T22:39:31.625230+0000","last_peered":"2024-04-24T22:39:31.625230+0000","last_clean":"2024-04-24T22:39:31.625230+0000","last_became_active":"2024-04-24T22:39:30.590157+0000","last_became_peered":"2024-04-24T22:39:30.590157+0000","last_unstale":"2024-04-24T22:39:31.625230+0000","last_undegraded":"2024-04-24T22:39:31.625230+0000","last_fullsized":"2024-04-24T22:39:31.625230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975204+0000","last_change":"2024-04-24T22:39:29.656600+0000","last_active":"2024-04-24T22:39:33.975204+0000","last_peered":"2024-04-24T22:39:33.975204+0000","last_clean":"2024-04-24T22:39:33.975204+0000","last_became_active":"2024-04-24T22:39:29.656286+0000","last_became_peered":"2024-04-24T22:39:29.656286+0000","last_unstale":"2024-04-24T22:39:33.975204+0000","last_undegraded":"2024-04-24T22:39:33.975204+0000","last_fullsized":"2024-04-24T22:39:33.975204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:33.605969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623587+0000","last_change":"2024-04-24T22:39:30.577213+0000","last_active":"2024-04-24T22:39:31.623587+0000","last_peered":"2024-04-24T22:39:31.623587+0000","last_clean":"2024-04-24T22:39:31.623587+0000","last_became_active":"2024-04-24T22:39:30.576883+0000","last_became_peered":"2024-04-24T22:39:30.576883+0000","last_unstale":"2024-04-24T22:39:31.623587+0000","last_undegraded":"2024-04-24T22:39:31.623587+0000","last_fullsized":"2024-04-24T22:39:31.623587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:16.984591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991575+0000","last_change":"2024-04-24T22:39:29.667082+0000","last_active":"2024-04-24T22:39:34.991575+0000","last_peered":"2024-04-24T22:39:34.991575+0000","last_clean":"2024-04-24T22:39:34.991575+0000","last_became_active":"2024-04-24T22:39:29.666631+0000","last_became_peered":"2024-04-24T22:39:29.666631+0000","last_unstale":"2024-04-24T22:39:34.991575+0000","last_undegraded":"2024-04-24T22:39:34.991575+0000","last_fullsized":"2024-04-24T22:39:34.991575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.001201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624978+0000","last_change":"2024-04-24T22:39:30.565848+0000","last_active":"2024-04-24T22:39:31.624978+0000","last_peered":"2024-04-24T22:39:31.624978+0000","last_clean":"2024-04-24T22:39:31.624978+0000","last_became_active":"2024-04-24T22:39:30.565533+0000","last_became_peered":"2024-04-24T22:39:30.565533+0000","last_unstale":"2024-04-24T22:39:31.624978+0000","last_undegraded":"2024-04-24T22:39:31.624978+0000","last_fullsized":"2024-04-24T22:39:31.624978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:02.406509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623407+0000","last_change":"2024-04-24T22:39:29.662575+0000","last_active":"2024-04-24T22:39:31.623407+0000","last_peered":"2024-04-24T22:39:31.623407+0000","last_clean":"2024-04-24T22:39:31.623407+0000","last_became_active":"2024-04-24T22:39:29.662273+0000","last_became_peered":"2024-04-24T22:39:29.662273+0000","last_unstale":"2024-04-24T22:39:31.623407+0000","last_undegraded":"2024-04-24T22:39:31.623407+0000","last_fullsized":"2024-04-24T22:39:31.623407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:42.661149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623689+0000","last_change":"2024-04-24T22:39:30.587813+0000","last_active":"2024-04-24T22:39:31.623689+0000","last_peered":"2024-04-24T22:39:31.623689+0000","last_clean":"2024-04-24T22:39:31.623689+0000","last_became_active":"2024-04-24T22:39:30.587533+0000","last_became_peered":"2024-04-24T22:39:30.587533+0000","last_unstale":"2024-04-24T22:39:31.623689+0000","last_undegraded":"2024-04-24T22:39:31.623689+0000","last_fullsized":"2024-04-24T22:39:31.623689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:27.042695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551034+0000","last_change":"2024-04-24T22:39:29.664843+0000","last_active":"2024-04-24T22:39:31.551034+0000","last_peered":"2024-04-24T22:39:31.551034+0000","last_clean":"2024-04-24T22:39:31.551034+0000","last_became_active":"2024-04-24T22:39:29.664542+0000","last_became_peered":"2024-04-24T22:39:29.664542+0000","last_unstale":"2024-04-24T22:39:31.551034+0000","last_undegraded":"2024-04-24T22:39:31.551034+0000","last_fullsized":"2024-04-24T22:39:31.551034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:17.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624484+0000","last_change":"2024-04-24T22:39:30.569753+0000","last_active":"2024-04-24T22:39:31.624484+0000","last_peered":"2024-04-24T22:39:31.624484+0000","last_clean":"2024-04-24T22:39:31.624484+0000","last_became_active":"2024-04-24T22:39:30.569433+0000","last_became_peered":"2024-04-24T22:39:30.569433+0000","last_unstale":"2024-04-24T22:39:31.624484+0000","last_undegraded":"2024-04-24T22:39:31.624484+0000","last_fullsized":"2024-04-24T22:39:31.624484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992834+0000","last_change":"2024-04-24T22:39:30.574076+0000","last_active":"2024-04-24T22:39:31.992834+0000","last_peered":"2024-04-24T22:39:31.992834+0000","last_clean":"2024-04-24T22:39:31.992834+0000","last_became_active":"2024-04-24T22:39:30.573769+0000","last_became_peered":"2024-04-24T22:39:30.573769+0000","last_unstale":"2024-04-24T22:39:31.992834+0000","last_undegraded":"2024-04-24T22:39:31.992834+0000","last_fullsized":"2024-04-24T22:39:31.992834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:55.456402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.552897+0000","last_change":"2024-04-24T22:39:30.570762+0000","last_active":"2024-04-24T22:39:31.552897+0000","last_peered":"2024-04-24T22:39:31.552897+0000","last_clean":"2024-04-24T22:39:31.552897+0000","last_became_active":"2024-04-24T22:39:30.570459+0000","last_became_peered":"2024-04-24T22:39:30.570459+0000","last_unstale":"2024-04-24T22:39:31.552897+0000","last_undegraded":"2024-04-24T22:39:31.552897+0000","last_fullsized":"2024-04-24T22:39:31.552897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.826752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.552910+0000","last_change":"2024-04-24T22:39:30.591414+0000","last_active":"2024-04-24T22:39:31.552910+0000","last_peered":"2024-04-24T22:39:31.552910+0000","last_clean":"2024-04-24T22:39:31.552910+0000","last_became_active":"2024-04-24T22:39:30.591135+0000","last_became_peered":"2024-04-24T22:39:30.591135+0000","last_unstale":"2024-04-24T22:39:31.552910+0000","last_undegraded":"2024-04-24T22:39:31.552910+0000","last_fullsized":"2024-04-24T22:39:31.552910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:58.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625095+0000","last_change":"2024-04-24T22:39:30.583821+0000","last_active":"2024-04-24T22:39:31.625095+0000","last_peered":"2024-04-24T22:39:31.625095+0000","last_clean":"2024-04-24T22:39:31.625095+0000","last_became_active":"2024-04-24T22:39:30.583476+0000","last_became_peered":"2024-04-24T22:39:30.583476+0000","last_unstale":"2024-04-24T22:39:31.625095+0000","last_undegraded":"2024-04-24T22:39:31.625095+0000","last_fullsized":"2024-04-24T22:39:31.625095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:08.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550780+0000","last_change":"2024-04-24T22:39:30.583955+0000","last_active":"2024-04-24T22:39:31.550780+0000","last_peered":"2024-04-24T22:39:31.550780+0000","last_clean":"2024-04-24T22:39:31.550780+0000","last_became_active":"2024-04-24T22:39:30.583632+0000","last_became_peered":"2024-04-24T22:39:30.583632+0000","last_unstale":"2024-04-24T22:39:31.550780+0000","last_undegraded":"2024-04-24T22:39:31.550780+0000","last_fullsized":"2024-04-24T22:39:31.550780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:44.150140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992910+0000","last_change":"2024-04-24T22:39:30.569989+0000","last_active":"2024-04-24T22:39:31.992910+0000","last_peered":"2024-04-24T22:39:31.992910+0000","last_clean":"2024-04-24T22:39:31.992910+0000","last_became_active":"2024-04-24T22:39:30.569680+0000","last_became_peered":"2024-04-24T22:39:30.569680+0000","last_unstale":"2024-04-24T22:39:31.992910+0000","last_undegraded":"2024-04-24T22:39:31.992910+0000","last_fullsized":"2024-04-24T22:39:31.992910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:55.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.553304+0000","last_change":"2024-04-24T22:39:30.572776+0000","last_active":"2024-04-24T22:39:31.553304+0000","last_peered":"2024-04-24T22:39:31.553304+0000","last_clean":"2024-04-24T22:39:31.553304+0000","last_became_active":"2024-04-24T22:39:30.572445+0000","last_became_peered":"2024-04-24T22:39:30.572445+0000","last_unstale":"2024-04-24T22:39:31.553304+0000","last_undegraded":"2024-04-24T22:39:31.553304+0000","last_fullsized":"2024-04-24T22:39:31.553304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:40.058377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624566+0000","last_change":"2024-04-24T22:39:30.587453+0000","last_active":"2024-04-24T22:39:31.624566+0000","last_peered":"2024-04-24T22:39:31.624566+0000","last_clean":"2024-04-24T22:39:31.624566+0000","last_became_active":"2024-04-24T22:39:30.586612+0000","last_became_peered":"2024-04-24T22:39:30.586612+0000","last_unstale":"2024-04-24T22:39:31.624566+0000","last_undegraded":"2024-04-24T22:39:31.624566+0000","last_fullsized":"2024-04-24T22:39:31.624566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:24.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623057+0000","last_change":"2024-04-24T22:39:30.592921+0000","last_active":"2024-04-24T22:39:31.623057+0000","last_peered":"2024-04-24T22:39:31.623057+0000","last_clean":"2024-04-24T22:39:31.623057+0000","last_became_active":"2024-04-24T22:39:30.592607+0000","last_became_peered":"2024-04-24T22:39:30.592607+0000","last_unstale":"2024-04-24T22:39:31.623057+0000","last_undegraded":"2024-04-24T22:39:31.623057+0000","last_fullsized":"2024-04-24T22:39:31.623057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:11.240743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624297+0000","last_change":"2024-04-24T22:39:30.586688+0000","last_active":"2024-04-24T22:39:31.624297+0000","last_peered":"2024-04-24T22:39:31.624297+0000","last_clean":"2024-04-24T22:39:31.624297+0000","last_became_active":"2024-04-24T22:39:30.586259+0000","last_became_peered":"2024-04-24T22:39:30.586259+0000","last_unstale":"2024-04-24T22:39:31.624297+0000","last_undegraded":"2024-04-24T22:39:31.624297+0000","last_fullsized":"2024-04-24T22:39:31.624297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:24.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550809+0000","last_change":"2024-04-24T22:39:30.591940+0000","last_active":"2024-04-24T22:39:31.550809+0000","last_peered":"2024-04-24T22:39:31.550809+0000","last_clean":"2024-04-24T22:39:31.550809+0000","last_became_active":"2024-04-24T22:39:30.591635+0000","last_became_peered":"2024-04-24T22:39:30.591635+0000","last_unstale":"2024-04-24T22:39:31.550809+0000","last_undegraded":"2024-04-24T22:39:31.550809+0000","last_fullsized":"2024-04-24T22:39:31.550809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:16.097707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551166+0000","last_change":"2024-04-24T22:39:30.588188+0000","last_active":"2024-04-24T22:39:31.551166+0000","last_peered":"2024-04-24T22:39:31.551166+0000","last_clean":"2024-04-24T22:39:31.551166+0000","last_became_active":"2024-04-24T22:39:30.587608+0000","last_became_peered":"2024-04-24T22:39:30.587608+0000","last_unstale":"2024-04-24T22:39:31.551166+0000","last_undegraded":"2024-04-24T22:39:31.551166+0000","last_fullsized":"2024-04-24T22:39:31.551166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:17.690009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992841+0000","last_change":"2024-04-24T22:39:30.567467+0000","last_active":"2024-04-24T22:39:31.992841+0000","last_peered":"2024-04-24T22:39:31.992841+0000","last_clean":"2024-04-24T22:39:31.992841+0000","last_became_active":"2024-04-24T22:39:30.567162+0000","last_became_peered":"2024-04-24T22:39:30.567162+0000","last_unstale":"2024-04-24T22:39:31.992841+0000","last_undegraded":"2024-04-24T22:39:31.992841+0000","last_fullsized":"2024-04-24T22:39:31.992841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:25.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":125,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":89610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1032079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1523712,"data_stored":1030681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":438728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":459416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":460766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1030365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":39543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:39.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 -- 172.21.15.111:0/3776364579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 msgr2=0x7f26d810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 --2- 172.21.15.111:0/3776364579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d810bc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009ca0 tx=0x7f26d002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 -- 172.21.15.111:0/3776364579 shutdown_connections 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 --2- 172.21.15.111:0/3776364579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d810bc00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 --2- 172.21.15.111:0/3776364579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81049b0 0x7f26d8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 --2- 172.21.15.111:0/3776364579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81037c0 0x7f26d8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 -- 172.21.15.111:0/3776364579 >> 172.21.15.111:0/3776364579 conn(0x7f26d80fed50 msgr2=0x7f26d8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 -- 172.21.15.111:0/3776364579 shutdown_connections 2024-04-24T22:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.101+0000 7f26d7e00640 1 -- 172.21.15.111:0/3776364579 wait complete. 2024-04-24T22:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.102+0000 7f26d7e00640 1 Processor -- start 2024-04-24T22:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.102+0000 7f26d7e00640 1 -- start start 2024-04-24T22:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 0x7f26d8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d81a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810f420 con 0x7f26d8105370 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810f2a0 con 0x7f26d81049b0 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810f5a0 con 0x7f26d81037c0 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d81a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45006/0 (socket says 172.21.15.111:45006) 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 0x7f26d8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d6a00640 1 -- 172.21.15.111:0/1324348357 learned_addr learned my addr 172.21.15.111:0/1324348357 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 -- 172.21.15.111:0/1324348357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 msgr2=0x7f26d8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 -- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 msgr2=0x7f26d81a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d81a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26cfe00640 1 -- 172.21.15.111:0/1324348357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d81a16a0 con 0x7f26d81049b0 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d7400640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d81a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.103+0000 7f26d6a00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26cfe00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 0x7f26d8199fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005fd0 tx=0x7f26c000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0009c30 con 0x7f26d81049b0 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81a1990 con 0x7f26d81049b0 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c0009dd0 con 0x7f26d81049b0 2024-04-24T22:39:40.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0010460 con 0x7f26d81049b0 2024-04-24T22:39:40.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.104+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81a1ee0 con 0x7f26d81049b0 2024-04-24T22:39:40.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.105+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d810aa80 con 0x7f26d81049b0 2024-04-24T22:39:40.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.108+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f26c0003f50 con 0x7f26d81049b0 2024-04-24T22:39:40.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.109+0000 7f26dca00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.109+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f26c00cefb0 con 0x7f26d81049b0 2024-04-24T22:39:40.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.109+0000 7f26d6a00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.110+0000 7f26d6a00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 0x7f26b80a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26c80058c0 tx=0x7f26c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:40.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.113+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26c0017020 con 0x7f26d81049b0 2024-04-24T22:39:40.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:40 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1060525163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:40.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:40 smithi111 ceph-mon[27746]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-04-24T22:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.310+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26d8061b80 con 0x7f26b80a2c20 2024-04-24T22:39:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.334+0000 7f26dca00640 1 -- 172.21.15.111:0/1324348357 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258705 (secure 0 0 0) 0x7f26d8061b80 con 0x7f26b80a2c20 2024-04-24T22:39:40.341 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:40.342 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.339+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 msgr2=0x7f26b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.339+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 0x7f26b80a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26c80058c0 tx=0x7f26c8005f90 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.339+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 msgr2=0x7f26d8199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.339+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 0x7f26d8199fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005fd0 tx=0x7f26c000dde0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 shutdown_connections 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8105370 0x7f26d81a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d81049b0 0x7f26d8199fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 --2- 172.21.15.111:0/1324348357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d81037c0 0x7f26d8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 >> 172.21.15.111:0/1324348357 conn(0x7f26d80fed50 msgr2=0x7f26d8107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 shutdown_connections 2024-04-24T22:39:40.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:40.340+0000 7f26d7e00640 1 -- 172.21.15.111:0/1324348357 wait complete. 2024-04-24T22:39:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-24T22:39:39.570477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595981,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":165,"num_read_kb":144,"num_write":306,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378512,"kb_used_data":12720,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563312,"statfs":{"total":1151940427776,"available":1151552831488,"internally_reserved":0,"allocated":13025280,"data_stored":7088096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":125,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.022838"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993005+0000","last_change":"2024-04-24T22:39:30.580615+0000","last_active":"2024-04-24T22:39:31.993005+0000","last_peered":"2024-04-24T22:39:31.993005+0000","last_clean":"2024-04-24T22:39:31.993005+0000","last_became_active":"2024-04-24T22:39:30.580299+0000","last_became_peered":"2024-04-24T22:39:30.580299+0000","last_unstale":"2024-04-24T22:39:31.993005+0000","last_undegraded":"2024-04-24T22:39:31.993005+0000","last_fullsized":"2024-04-24T22:39:31.993005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.364115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625609+0000","last_change":"2024-04-24T22:39:30.588495+0000","last_active":"2024-04-24T22:39:31.625609+0000","last_peered":"2024-04-24T22:39:31.625609+0000","last_clean":"2024-04-24T22:39:31.625609+0000","last_became_active":"2024-04-24T22:39:30.585588+0000","last_became_peered":"2024-04-24T22:39:30.585588+0000","last_unstale":"2024-04-24T22:39:31.625609+0000","last_undegraded":"2024-04-24T22:39:31.625609+0000","last_fullsized":"2024-04-24T22:39:31.625609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.894832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625461+0000","last_change":"2024-04-24T22:39:30.585139+0000","last_active":"2024-04-24T22:39:31.625461+0000","last_peered":"2024-04-24T22:39:31.625461+0000","last_clean":"2024-04-24T22:39:31.625461+0000","last_became_active":"2024-04-24T22:39:30.583299+0000","last_became_peered":"2024-04-24T22:39:30.583299+0000","last_unstale":"2024-04-24T22:39:31.625461+0000","last_undegraded":"2024-04-24T22:39:31.625461+0000","last_fullsized":"2024-04-24T22:39:31.625461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:20.941383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624460+0000","last_change":"2024-04-24T22:39:30.566540+0000","last_active":"2024-04-24T22:39:31.624460+0000","last_peered":"2024-04-24T22:39:31.624460+0000","last_clean":"2024-04-24T22:39:31.624460+0000","last_became_active":"2024-04-24T22:39:30.566195+0000","last_became_peered":"2024-04-24T22:39:30.566195+0000","last_unstale":"2024-04-24T22:39:31.624460+0000","last_undegraded":"2024-04-24T22:39:31.624460+0000","last_fullsized":"2024-04-24T22:39:31.624460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:24.538153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623321+0000","last_change":"2024-04-24T22:39:30.591842+0000","last_active":"2024-04-24T22:39:31.623321+0000","last_peered":"2024-04-24T22:39:31.623321+0000","last_clean":"2024-04-24T22:39:31.623321+0000","last_became_active":"2024-04-24T22:39:30.591579+0000","last_became_peered":"2024-04-24T22:39:30.591579+0000","last_unstale":"2024-04-24T22:39:31.623321+0000","last_undegraded":"2024-04-24T22:39:31.623321+0000","last_fullsized":"2024-04-24T22:39:31.623321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:48.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623682+0000","last_change":"2024-04-24T22:39:30.577689+0000","last_active":"2024-04-24T22:39:31.623682+0000","last_peered":"2024-04-24T22:39:31.623682+0000","last_clean":"2024-04-24T22:39:31.623682+0000","last_became_active":"2024-04-24T22:39:30.577362+0000","last_became_peered":"2024-04-24T22:39:30.577362+0000","last_unstale":"2024-04-24T22:39:31.623682+0000","last_undegraded":"2024-04-24T22:39:31.623682+0000","last_fullsized":"2024-04-24T22:39:31.623682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:52.808898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623234+0000","last_change":"2024-04-24T22:39:30.576400+0000","last_active":"2024-04-24T22:39:31.623234+0000","last_peered":"2024-04-24T22:39:31.623234+0000","last_clean":"2024-04-24T22:39:31.623234+0000","last_became_active":"2024-04-24T22:39:30.576141+0000","last_became_peered":"2024-04-24T22:39:30.576141+0000","last_unstale":"2024-04-24T22:39:31.623234+0000","last_undegraded":"2024-04-24T22:39:31.623234+0000","last_fullsized":"2024-04-24T22:39:31.623234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623755+0000","last_change":"2024-04-24T22:39:30.594496+0000","last_active":"2024-04-24T22:39:31.623755+0000","last_peered":"2024-04-24T22:39:31.623755+0000","last_clean":"2024-04-24T22:39:31.623755+0000","last_became_active":"2024-04-24T22:39:30.589541+0000","last_became_peered":"2024-04-24T22:39:30.589541+0000","last_unstale":"2024-04-24T22:39:31.623755+0000","last_undegraded":"2024-04-24T22:39:31.623755+0000","last_fullsized":"2024-04-24T22:39:31.623755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:50.593142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551116+0000","last_change":"2024-04-24T22:39:30.567922+0000","last_active":"2024-04-24T22:39:31.551116+0000","last_peered":"2024-04-24T22:39:31.551116+0000","last_clean":"2024-04-24T22:39:31.551116+0000","last_became_active":"2024-04-24T22:39:30.567470+0000","last_became_peered":"2024-04-24T22:39:30.567470+0000","last_unstale":"2024-04-24T22:39:31.551116+0000","last_undegraded":"2024-04-24T22:39:31.551116+0000","last_fullsized":"2024-04-24T22:39:31.551116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:11.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623972+0000","last_change":"2024-04-24T22:39:30.566716+0000","last_active":"2024-04-24T22:39:31.623972+0000","last_peered":"2024-04-24T22:39:31.623972+0000","last_clean":"2024-04-24T22:39:31.623972+0000","last_became_active":"2024-04-24T22:39:30.566192+0000","last_became_peered":"2024-04-24T22:39:30.566192+0000","last_unstale":"2024-04-24T22:39:31.623972+0000","last_undegraded":"2024-04-24T22:39:31.623972+0000","last_fullsized":"2024-04-24T22:39:31.623972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:36.364984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624739+0000","last_change":"2024-04-24T22:39:30.571679+0000","last_active":"2024-04-24T22:39:31.624739+0000","last_peered":"2024-04-24T22:39:31.624739+0000","last_clean":"2024-04-24T22:39:31.624739+0000","last_became_active":"2024-04-24T22:39:30.571375+0000","last_became_peered":"2024-04-24T22:39:30.571375+0000","last_unstale":"2024-04-24T22:39:31.624739+0000","last_undegraded":"2024-04-24T22:39:31.624739+0000","last_fullsized":"2024-04-24T22:39:31.624739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:47.219082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624257+0000","last_change":"2024-04-24T22:39:30.565158+0000","last_active":"2024-04-24T22:39:31.624257+0000","last_peered":"2024-04-24T22:39:31.624257+0000","last_clean":"2024-04-24T22:39:31.624257+0000","last_became_active":"2024-04-24T22:39:30.564808+0000","last_became_peered":"2024-04-24T22:39:30.564808+0000","last_unstale":"2024-04-24T22:39:31.624257+0000","last_undegraded":"2024-04-24T22:39:31.624257+0000","last_fullsized":"2024-04-24T22:39:31.624257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:33.534801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551139+0000","last_change":"2024-04-24T22:39:30.561125+0000","last_active":"2024-04-24T22:39:31.551139+0000","last_peered":"2024-04-24T22:39:31.551139+0000","last_clean":"2024-04-24T22:39:31.551139+0000","last_became_active":"2024-04-24T22:39:30.559473+0000","last_became_peered":"2024-04-24T22:39:30.559473+0000","last_unstale":"2024-04-24T22:39:31.551139+0000","last_undegraded":"2024-04-24T22:39:31.551139+0000","last_fullsized":"2024-04-24T22:39:31.551139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:09.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624224+0000","last_change":"2024-04-24T22:39:30.582371+0000","last_active":"2024-04-24T22:39:31.624224+0000","last_peered":"2024-04-24T22:39:31.624224+0000","last_clean":"2024-04-24T22:39:31.624224+0000","last_became_active":"2024-04-24T22:39:30.582002+0000","last_became_peered":"2024-04-24T22:39:30.582002+0000","last_unstale":"2024-04-24T22:39:31.624224+0000","last_undegraded":"2024-04-24T22:39:31.624224+0000","last_fullsized":"2024-04-24T22:39:31.624224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:11.812461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623952+0000","last_change":"2024-04-24T22:39:30.581249+0000","last_active":"2024-04-24T22:39:31.623952+0000","last_peered":"2024-04-24T22:39:31.623952+0000","last_clean":"2024-04-24T22:39:31.623952+0000","last_became_active":"2024-04-24T22:39:30.580820+0000","last_became_peered":"2024-04-24T22:39:30.580820+0000","last_unstale":"2024-04-24T22:39:31.623952+0000","last_undegraded":"2024-04-24T22:39:31.623952+0000","last_fullsized":"2024-04-24T22:39:31.623952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:58.940926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550624+0000","last_change":"2024-04-24T22:39:30.578310+0000","last_active":"2024-04-24T22:39:31.550624+0000","last_peered":"2024-04-24T22:39:31.550624+0000","last_clean":"2024-04-24T22:39:31.550624+0000","last_became_active":"2024-04-24T22:39:30.577917+0000","last_became_peered":"2024-04-24T22:39:30.577917+0000","last_unstale":"2024-04-24T22:39:31.550624+0000","last_undegraded":"2024-04-24T22:39:31.550624+0000","last_fullsized":"2024-04-24T22:39:31.550624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:51.766196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993166+0000","last_change":"2024-04-24T22:39:30.580939+0000","last_active":"2024-04-24T22:39:31.993166+0000","last_peered":"2024-04-24T22:39:31.993166+0000","last_clean":"2024-04-24T22:39:31.993166+0000","last_became_active":"2024-04-24T22:39:30.580583+0000","last_became_peered":"2024-04-24T22:39:30.580583+0000","last_unstale":"2024-04-24T22:39:31.993166+0000","last_undegraded":"2024-04-24T22:39:31.993166+0000","last_fullsized":"2024-04-24T22:39:31.993166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.891103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626593+0000","last_change":"2024-04-24T22:39:30.577933+0000","last_active":"2024-04-24T22:39:31.626593+0000","last_peered":"2024-04-24T22:39:31.626593+0000","last_clean":"2024-04-24T22:39:31.626593+0000","last_became_active":"2024-04-24T22:39:30.577545+0000","last_became_peered":"2024-04-24T22:39:30.577545+0000","last_unstale":"2024-04-24T22:39:31.626593+0000","last_undegraded":"2024-04-24T22:39:31.626593+0000","last_fullsized":"2024-04-24T22:39:31.626593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:26.876281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.553246+0000","last_change":"2024-04-24T22:39:30.586170+0000","last_active":"2024-04-24T22:39:31.553246+0000","last_peered":"2024-04-24T22:39:31.553246+0000","last_clean":"2024-04-24T22:39:31.553246+0000","last_became_active":"2024-04-24T22:39:30.585922+0000","last_became_peered":"2024-04-24T22:39:30.585922+0000","last_unstale":"2024-04-24T22:39:31.553246+0000","last_undegraded":"2024-04-24T22:39:31.553246+0000","last_fullsized":"2024-04-24T22:39:31.553246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:53.166725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.992381+0000","last_change":"2024-04-24T22:39:29.660103+0000","last_active":"2024-04-24T22:39:34.992381+0000","last_peered":"2024-04-24T22:39:34.992381+0000","last_clean":"2024-04-24T22:39:34.992381+0000","last_became_active":"2024-04-24T22:39:29.659809+0000","last_became_peered":"2024-04-24T22:39:29.659809+0000","last_unstale":"2024-04-24T22:39:34.992381+0000","last_undegraded":"2024-04-24T22:39:34.992381+0000","last_fullsized":"2024-04-24T22:39:34.992381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:40.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624143+0000","last_change":"2024-04-24T22:39:30.587041+0000","last_active":"2024-04-24T22:39:31.624143+0000","last_peered":"2024-04-24T22:39:31.624143+0000","last_clean":"2024-04-24T22:39:31.624143+0000","last_became_active":"2024-04-24T22:39:30.586568+0000","last_became_peered":"2024-04-24T22:39:30.586568+0000","last_unstale":"2024-04-24T22:39:31.624143+0000","last_undegraded":"2024-04-24T22:39:31.624143+0000","last_fullsized":"2024-04-24T22:39:31.624143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:13.305790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624532+0000","last_change":"2024-04-24T22:39:29.659591+0000","last_active":"2024-04-24T22:39:31.624532+0000","last_peered":"2024-04-24T22:39:31.624532+0000","last_clean":"2024-04-24T22:39:31.624532+0000","last_became_active":"2024-04-24T22:39:29.659281+0000","last_became_peered":"2024-04-24T22:39:29.659281+0000","last_unstale":"2024-04-24T22:39:31.624532+0000","last_undegraded":"2024-04-24T22:39:31.624532+0000","last_fullsized":"2024-04-24T22:39:31.624532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:07.909300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626309+0000","last_change":"2024-04-24T22:39:30.589041+0000","last_active":"2024-04-24T22:39:31.626309+0000","last_peered":"2024-04-24T22:39:31.626309+0000","last_clean":"2024-04-24T22:39:31.626309+0000","last_became_active":"2024-04-24T22:39:30.588706+0000","last_became_peered":"2024-04-24T22:39:30.588706+0000","last_unstale":"2024-04-24T22:39:31.626309+0000","last_undegraded":"2024-04-24T22:39:31.626309+0000","last_fullsized":"2024-04-24T22:39:31.626309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:19.594236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.986772+0000","last_change":"2024-04-24T22:39:29.659280+0000","last_active":"2024-04-24T22:39:34.986772+0000","last_peered":"2024-04-24T22:39:34.986772+0000","last_clean":"2024-04-24T22:39:34.986772+0000","last_became_active":"2024-04-24T22:39:29.658922+0000","last_became_peered":"2024-04-24T22:39:29.658922+0000","last_unstale":"2024-04-24T22:39:34.986772+0000","last_undegraded":"2024-04-24T22:39:34.986772+0000","last_fullsized":"2024-04-24T22:39:34.986772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:58.157675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550771+0000","last_change":"2024-04-24T22:39:30.573769+0000","last_active":"2024-04-24T22:39:31.550771+0000","last_peered":"2024-04-24T22:39:31.550771+0000","last_clean":"2024-04-24T22:39:31.550771+0000","last_became_active":"2024-04-24T22:39:30.573405+0000","last_became_peered":"2024-04-24T22:39:30.573405+0000","last_unstale":"2024-04-24T22:39:31.550771+0000","last_undegraded":"2024-04-24T22:39:31.550771+0000","last_fullsized":"2024-04-24T22:39:31.550771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:08.919808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623929+0000","last_change":"2024-04-24T22:39:29.653759+0000","last_active":"2024-04-24T22:39:31.623929+0000","last_peered":"2024-04-24T22:39:31.623929+0000","last_clean":"2024-04-24T22:39:31.623929+0000","last_became_active":"2024-04-24T22:39:29.653060+0000","last_became_peered":"2024-04-24T22:39:29.653060+0000","last_unstale":"2024-04-24T22:39:31.623929+0000","last_undegraded":"2024-04-24T22:39:31.623929+0000","last_fullsized":"2024-04-24T22:39:31.623929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:02.503014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623517+0000","last_change":"2024-04-24T22:39:30.563070+0000","last_active":"2024-04-24T22:39:31.623517+0000","last_peered":"2024-04-24T22:39:31.623517+0000","last_clean":"2024-04-24T22:39:31.623517+0000","last_became_active":"2024-04-24T22:39:30.562736+0000","last_became_peered":"2024-04-24T22:39:30.562736+0000","last_unstale":"2024-04-24T22:39:31.623517+0000","last_undegraded":"2024-04-24T22:39:31.623517+0000","last_fullsized":"2024-04-24T22:39:31.623517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:15.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000955+0000","last_change":"2024-04-24T22:39:29.657659+0000","last_active":"2024-04-24T22:39:36.000955+0000","last_peered":"2024-04-24T22:39:36.000955+0000","last_clean":"2024-04-24T22:39:36.000955+0000","last_became_active":"2024-04-24T22:39:29.657237+0000","last_became_peered":"2024-04-24T22:39:29.657237+0000","last_unstale":"2024-04-24T22:39:36.000955+0000","last_undegraded":"2024-04-24T22:39:36.000955+0000","last_fullsized":"2024-04-24T22:39:36.000955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:11.376959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623743+0000","last_change":"2024-04-24T22:39:30.577542+0000","last_active":"2024-04-24T22:39:31.623743+0000","last_peered":"2024-04-24T22:39:31.623743+0000","last_clean":"2024-04-24T22:39:31.623743+0000","last_became_active":"2024-04-24T22:39:30.577194+0000","last_became_peered":"2024-04-24T22:39:30.577194+0000","last_unstale":"2024-04-24T22:39:31.623743+0000","last_undegraded":"2024-04-24T22:39:31.623743+0000","last_fullsized":"2024-04-24T22:39:31.623743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:26.000787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991451+0000","last_change":"2024-04-24T22:39:29.671709+0000","last_active":"2024-04-24T22:39:34.991451+0000","last_peered":"2024-04-24T22:39:34.991451+0000","last_clean":"2024-04-24T22:39:34.991451+0000","last_became_active":"2024-04-24T22:39:29.671306+0000","last_became_peered":"2024-04-24T22:39:29.671306+0000","last_unstale":"2024-04-24T22:39:34.991451+0000","last_undegraded":"2024-04-24T22:39:34.991451+0000","last_fullsized":"2024-04-24T22:39:34.991451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:33.833217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623268+0000","last_change":"2024-04-24T22:39:30.583720+0000","last_active":"2024-04-24T22:39:31.623268+0000","last_peered":"2024-04-24T22:39:31.623268+0000","last_clean":"2024-04-24T22:39:31.623268+0000","last_became_active":"2024-04-24T22:39:30.583468+0000","last_became_peered":"2024-04-24T22:39:30.583468+0000","last_unstale":"2024-04-24T22:39:31.623268+0000","last_undegraded":"2024-04-24T22:39:31.623268+0000","last_fullsized":"2024-04-24T22:39:31.623268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:07.679284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.992195+0000","last_change":"2024-04-24T22:39:29.669100+0000","last_active":"2024-04-24T22:39:34.992195+0000","last_peered":"2024-04-24T22:39:34.992195+0000","last_clean":"2024-04-24T22:39:34.992195+0000","last_became_active":"2024-04-24T22:39:29.668773+0000","last_became_peered":"2024-04-24T22:39:29.668773+0000","last_unstale":"2024-04-24T22:39:34.992195+0000","last_undegraded":"2024-04-24T22:39:34.992195+0000","last_fullsized":"2024-04-24T22:39:34.992195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:56.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624052+0000","last_change":"2024-04-24T22:39:30.594562+0000","last_active":"2024-04-24T22:39:31.624052+0000","last_peered":"2024-04-24T22:39:31.624052+0000","last_clean":"2024-04-24T22:39:31.624052+0000","last_became_active":"2024-04-24T22:39:30.589928+0000","last_became_peered":"2024-04-24T22:39:30.589928+0000","last_unstale":"2024-04-24T22:39:31.624052+0000","last_undegraded":"2024-04-24T22:39:31.624052+0000","last_fullsized":"2024-04-24T22:39:31.624052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:14.951690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623174+0000","last_change":"2024-04-24T22:39:29.658615+0000","last_active":"2024-04-24T22:39:31.623174+0000","last_peered":"2024-04-24T22:39:31.623174+0000","last_clean":"2024-04-24T22:39:31.623174+0000","last_became_active":"2024-04-24T22:39:29.658178+0000","last_became_peered":"2024-04-24T22:39:29.658178+0000","last_unstale":"2024-04-24T22:39:31.623174+0000","last_undegraded":"2024-04-24T22:39:31.623174+0000","last_fullsized":"2024-04-24T22:39:31.623174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.821509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625744+0000","last_change":"2024-04-24T22:39:30.592142+0000","last_active":"2024-04-24T22:39:31.625744+0000","last_peered":"2024-04-24T22:39:31.625744+0000","last_clean":"2024-04-24T22:39:31.625744+0000","last_became_active":"2024-04-24T22:39:30.591417+0000","last_became_peered":"2024-04-24T22:39:30.591417+0000","last_unstale":"2024-04-24T22:39:31.625744+0000","last_undegraded":"2024-04-24T22:39:31.625744+0000","last_fullsized":"2024-04-24T22:39:31.625744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:55.818795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.978374+0000","last_change":"2024-04-24T22:39:29.662513+0000","last_active":"2024-04-24T22:39:33.978374+0000","last_peered":"2024-04-24T22:39:33.978374+0000","last_clean":"2024-04-24T22:39:33.978374+0000","last_became_active":"2024-04-24T22:39:29.662223+0000","last_became_peered":"2024-04-24T22:39:29.662223+0000","last_unstale":"2024-04-24T22:39:33.978374+0000","last_undegraded":"2024-04-24T22:39:33.978374+0000","last_fullsized":"2024-04-24T22:39:33.978374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.012595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550956+0000","last_change":"2024-04-24T22:39:30.577594+0000","last_active":"2024-04-24T22:39:31.550956+0000","last_peered":"2024-04-24T22:39:31.550956+0000","last_clean":"2024-04-24T22:39:31.550956+0000","last_became_active":"2024-04-24T22:39:30.577254+0000","last_became_peered":"2024-04-24T22:39:30.577254+0000","last_unstale":"2024-04-24T22:39:31.550956+0000","last_undegraded":"2024-04-24T22:39:31.550956+0000","last_fullsized":"2024-04-24T22:39:31.550956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:47.926878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.973884+0000","last_change":"2024-04-24T22:39:29.659898+0000","last_active":"2024-04-24T22:39:33.973884+0000","last_peered":"2024-04-24T22:39:33.973884+0000","last_clean":"2024-04-24T22:39:33.973884+0000","last_became_active":"2024-04-24T22:39:29.659582+0000","last_became_peered":"2024-04-24T22:39:29.659582+0000","last_unstale":"2024-04-24T22:39:33.973884+0000","last_undegraded":"2024-04-24T22:39:33.973884+0000","last_fullsized":"2024-04-24T22:39:33.973884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626164+0000","last_change":"2024-04-24T22:39:30.588271+0000","last_active":"2024-04-24T22:39:31.626164+0000","last_peered":"2024-04-24T22:39:31.626164+0000","last_clean":"2024-04-24T22:39:31.626164+0000","last_became_active":"2024-04-24T22:39:30.578846+0000","last_became_peered":"2024-04-24T22:39:30.578846+0000","last_unstale":"2024-04-24T22:39:31.626164+0000","last_undegraded":"2024-04-24T22:39:31.626164+0000","last_fullsized":"2024-04-24T22:39:31.626164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:23.807550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.976436+0000","last_change":"2024-04-24T22:39:29.654708+0000","last_active":"2024-04-24T22:39:33.976436+0000","last_peered":"2024-04-24T22:39:33.976436+0000","last_clean":"2024-04-24T22:39:33.976436+0000","last_became_active":"2024-04-24T22:39:29.654290+0000","last_became_peered":"2024-04-24T22:39:29.654290+0000","last_unstale":"2024-04-24T22:39:33.976436+0000","last_undegraded":"2024-04-24T22:39:33.976436+0000","last_fullsized":"2024-04-24T22:39:33.976436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:29.445700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550820+0000","last_change":"2024-04-24T22:39:30.569436+0000","last_active":"2024-04-24T22:39:31.550820+0000","last_peered":"2024-04-24T22:39:31.550820+0000","last_clean":"2024-04-24T22:39:31.550820+0000","last_became_active":"2024-04-24T22:39:30.569104+0000","last_became_peered":"2024-04-24T22:39:30.569104+0000","last_unstale":"2024-04-24T22:39:31.550820+0000","last_undegraded":"2024-04-24T22:39:31.550820+0000","last_fullsized":"2024-04-24T22:39:31.550820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:30.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000867+0000","last_change":"2024-04-24T22:39:29.657883+0000","last_active":"2024-04-24T22:39:36.000867+0000","last_peered":"2024-04-24T22:39:36.000867+0000","last_clean":"2024-04-24T22:39:36.000867+0000","last_became_active":"2024-04-24T22:39:29.657496+0000","last_became_peered":"2024-04-24T22:39:29.657496+0000","last_unstale":"2024-04-24T22:39:36.000867+0000","last_undegraded":"2024-04-24T22:39:36.000867+0000","last_fullsized":"2024-04-24T22:39:36.000867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:51:32.037937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624173+0000","last_change":"2024-04-24T22:39:30.569744+0000","last_active":"2024-04-24T22:39:31.624173+0000","last_peered":"2024-04-24T22:39:31.624173+0000","last_clean":"2024-04-24T22:39:31.624173+0000","last_became_active":"2024-04-24T22:39:30.569392+0000","last_became_peered":"2024-04-24T22:39:30.569392+0000","last_unstale":"2024-04-24T22:39:31.624173+0000","last_undegraded":"2024-04-24T22:39:31.624173+0000","last_fullsized":"2024-04-24T22:39:31.624173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:09.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.993575+0000","last_change":"2024-04-24T22:39:29.657646+0000","last_active":"2024-04-24T22:39:34.993575+0000","last_peered":"2024-04-24T22:39:34.993575+0000","last_clean":"2024-04-24T22:39:34.993575+0000","last_became_active":"2024-04-24T22:39:29.657302+0000","last_became_peered":"2024-04-24T22:39:29.657302+0000","last_unstale":"2024-04-24T22:39:34.993575+0000","last_undegraded":"2024-04-24T22:39:34.993575+0000","last_fullsized":"2024-04-24T22:39:34.993575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:01.449788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551026+0000","last_change":"2024-04-24T22:39:30.591911+0000","last_active":"2024-04-24T22:39:31.551026+0000","last_peered":"2024-04-24T22:39:31.551026+0000","last_clean":"2024-04-24T22:39:31.551026+0000","last_became_active":"2024-04-24T22:39:30.591520+0000","last_became_peered":"2024-04-24T22:39:30.591520+0000","last_unstale":"2024-04-24T22:39:31.551026+0000","last_undegraded":"2024-04-24T22:39:31.551026+0000","last_fullsized":"2024-04-24T22:39:31.551026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:02.545529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975200+0000","last_change":"2024-04-24T22:39:29.657561+0000","last_active":"2024-04-24T22:39:33.975200+0000","last_peered":"2024-04-24T22:39:33.975200+0000","last_clean":"2024-04-24T22:39:33.975200+0000","last_became_active":"2024-04-24T22:39:29.657261+0000","last_became_peered":"2024-04-24T22:39:29.657261+0000","last_unstale":"2024-04-24T22:39:33.975200+0000","last_undegraded":"2024-04-24T22:39:33.975200+0000","last_fullsized":"2024-04-24T22:39:33.975200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:11.856640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624016+0000","last_change":"2024-04-24T22:39:30.577405+0000","last_active":"2024-04-24T22:39:31.624016+0000","last_peered":"2024-04-24T22:39:31.624016+0000","last_clean":"2024-04-24T22:39:31.624016+0000","last_became_active":"2024-04-24T22:39:30.576980+0000","last_became_peered":"2024-04-24T22:39:30.576980+0000","last_unstale":"2024-04-24T22:39:31.624016+0000","last_undegraded":"2024-04-24T22:39:31.624016+0000","last_fullsized":"2024-04-24T22:39:31.624016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:00.145497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:37.005193+0000","last_change":"2024-04-24T22:39:29.661054+0000","last_active":"2024-04-24T22:39:37.005193+0000","last_peered":"2024-04-24T22:39:37.005193+0000","last_clean":"2024-04-24T22:39:37.005193+0000","last_became_active":"2024-04-24T22:39:29.660638+0000","last_became_peered":"2024-04-24T22:39:29.660638+0000","last_unstale":"2024-04-24T22:39:37.005193+0000","last_undegraded":"2024-04-24T22:39:37.005193+0000","last_fullsized":"2024-04-24T22:39:37.005193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:51.891654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623158+0000","last_change":"2024-04-24T22:39:30.570407+0000","last_active":"2024-04-24T22:39:31.623158+0000","last_peered":"2024-04-24T22:39:31.623158+0000","last_clean":"2024-04-24T22:39:31.623158+0000","last_became_active":"2024-04-24T22:39:30.570070+0000","last_became_peered":"2024-04-24T22:39:30.570070+0000","last_unstale":"2024-04-24T22:39:31.623158+0000","last_undegraded":"2024-04-24T22:39:31.623158+0000","last_fullsized":"2024-04-24T22:39:31.623158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991910+0000","last_change":"2024-04-24T22:39:29.667020+0000","last_active":"2024-04-24T22:39:34.991910+0000","last_peered":"2024-04-24T22:39:34.991910+0000","last_clean":"2024-04-24T22:39:34.991910+0000","last_became_active":"2024-04-24T22:39:29.666427+0000","last_became_peered":"2024-04-24T22:39:29.666427+0000","last_unstale":"2024-04-24T22:39:34.991910+0000","last_undegraded":"2024-04-24T22:39:34.991910+0000","last_fullsized":"2024-04-24T22:39:34.991910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:49.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623048+0000","last_change":"2024-04-24T22:39:30.562092+0000","last_active":"2024-04-24T22:39:31.623048+0000","last_peered":"2024-04-24T22:39:31.623048+0000","last_clean":"2024-04-24T22:39:31.623048+0000","last_became_active":"2024-04-24T22:39:30.561704+0000","last_became_peered":"2024-04-24T22:39:30.561704+0000","last_unstale":"2024-04-24T22:39:31.623048+0000","last_undegraded":"2024-04-24T22:39:31.623048+0000","last_fullsized":"2024-04-24T22:39:31.623048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.996837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624054+0000","last_change":"2024-04-24T22:39:29.655612+0000","last_active":"2024-04-24T22:39:31.624054+0000","last_peered":"2024-04-24T22:39:31.624054+0000","last_clean":"2024-04-24T22:39:31.624054+0000","last_became_active":"2024-04-24T22:39:29.655212+0000","last_became_peered":"2024-04-24T22:39:29.655212+0000","last_unstale":"2024-04-24T22:39:31.624054+0000","last_undegraded":"2024-04-24T22:39:31.624054+0000","last_fullsized":"2024-04-24T22:39:31.624054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.414774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.993019+0000","last_change":"2024-04-24T22:39:30.571705+0000","last_active":"2024-04-24T22:39:31.993019+0000","last_peered":"2024-04-24T22:39:31.993019+0000","last_clean":"2024-04-24T22:39:31.993019+0000","last_became_active":"2024-04-24T22:39:30.571158+0000","last_became_peered":"2024-04-24T22:39:30.571158+0000","last_unstale":"2024-04-24T22:39:31.993019+0000","last_undegraded":"2024-04-24T22:39:31.993019+0000","last_fullsized":"2024-04-24T22:39:31.993019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:45:47.935928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.626544+0000","last_change":"2024-04-24T22:39:29.661052+0000","last_active":"2024-04-24T22:39:31.626544+0000","last_peered":"2024-04-24T22:39:31.626544+0000","last_clean":"2024-04-24T22:39:31.626544+0000","last_became_active":"2024-04-24T22:39:29.660716+0000","last_became_peered":"2024-04-24T22:39:29.660716+0000","last_unstale":"2024-04-24T22:39:31.626544+0000","last_undegraded":"2024-04-24T22:39:31.626544+0000","last_fullsized":"2024-04-24T22:39:31.626544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:52.016503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624855+0000","last_change":"2024-04-24T22:39:30.585190+0000","last_active":"2024-04-24T22:39:31.624855+0000","last_peered":"2024-04-24T22:39:31.624855+0000","last_clean":"2024-04-24T22:39:31.624855+0000","last_became_active":"2024-04-24T22:39:30.584846+0000","last_became_peered":"2024-04-24T22:39:30.584846+0000","last_unstale":"2024-04-24T22:39:31.624855+0000","last_undegraded":"2024-04-24T22:39:31.624855+0000","last_fullsized":"2024-04-24T22:39:31.624855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:07.094297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.795109+0000","last_change":"2024-04-24T22:39:29.671686+0000","last_active":"2024-04-24T22:39:31.795109+0000","last_peered":"2024-04-24T22:39:31.795109+0000","last_clean":"2024-04-24T22:39:31.795109+0000","last_became_active":"2024-04-24T22:39:29.671338+0000","last_became_peered":"2024-04-24T22:39:29.671338+0000","last_unstale":"2024-04-24T22:39:31.795109+0000","last_undegraded":"2024-04-24T22:39:31.795109+0000","last_fullsized":"2024-04-24T22:39:31.795109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:13.709429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623523+0000","last_change":"2024-04-24T22:39:30.585560+0000","last_active":"2024-04-24T22:39:31.623523+0000","last_peered":"2024-04-24T22:39:31.623523+0000","last_clean":"2024-04-24T22:39:31.623523+0000","last_became_active":"2024-04-24T22:39:30.585251+0000","last_became_peered":"2024-04-24T22:39:30.585251+0000","last_unstale":"2024-04-24T22:39:31.623523+0000","last_undegraded":"2024-04-24T22:39:31.623523+0000","last_fullsized":"2024-04-24T22:39:31.623523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:29.680556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623315+0000","last_change":"2024-04-24T22:39:30.569105+0000","last_active":"2024-04-24T22:39:31.623315+0000","last_peered":"2024-04-24T22:39:31.623315+0000","last_clean":"2024-04-24T22:39:31.623315+0000","last_became_active":"2024-04-24T22:39:30.566878+0000","last_became_peered":"2024-04-24T22:39:30.566878+0000","last_unstale":"2024-04-24T22:39:31.623315+0000","last_undegraded":"2024-04-24T22:39:31.623315+0000","last_fullsized":"2024-04-24T22:39:31.623315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.334294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.002064+0000","last_change":"2024-04-24T22:39:29.674334+0000","last_active":"2024-04-24T22:39:36.002064+0000","last_peered":"2024-04-24T22:39:36.002064+0000","last_clean":"2024-04-24T22:39:36.002064+0000","last_became_active":"2024-04-24T22:39:29.673968+0000","last_became_peered":"2024-04-24T22:39:29.673968+0000","last_unstale":"2024-04-24T22:39:36.002064+0000","last_undegraded":"2024-04-24T22:39:36.002064+0000","last_fullsized":"2024-04-24T22:39:36.002064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:10.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624791+0000","last_change":"2024-04-24T22:39:30.579663+0000","last_active":"2024-04-24T22:39:31.624791+0000","last_peered":"2024-04-24T22:39:31.624791+0000","last_clean":"2024-04-24T22:39:31.624791+0000","last_became_active":"2024-04-24T22:39:30.578730+0000","last_became_peered":"2024-04-24T22:39:30.578730+0000","last_unstale":"2024-04-24T22:39:31.624791+0000","last_undegraded":"2024-04-24T22:39:31.624791+0000","last_fullsized":"2024-04-24T22:39:31.624791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:25.604081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:36.000002+0000","last_change":"2024-04-24T22:39:29.655316+0000","last_active":"2024-04-24T22:39:36.000002+0000","last_peered":"2024-04-24T22:39:36.000002+0000","last_clean":"2024-04-24T22:39:36.000002+0000","last_became_active":"2024-04-24T22:39:29.654975+0000","last_became_peered":"2024-04-24T22:39:29.654975+0000","last_unstale":"2024-04-24T22:39:36.000002+0000","last_undegraded":"2024-04-24T22:39:36.000002+0000","last_fullsized":"2024-04-24T22:39:36.000002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:20.461744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623223+0000","last_change":"2024-04-24T22:38:09.915368+0000","last_active":"2024-04-24T22:39:31.623223+0000","last_peered":"2024-04-24T22:39:31.623223+0000","last_clean":"2024-04-24T22:39:31.623223+0000","last_became_active":"2024-04-24T22:38:09.914879+0000","last_became_peered":"2024-04-24T22:38:09.914879+0000","last_unstale":"2024-04-24T22:39:31.623223+0000","last_undegraded":"2024-04-24T22:39:31.623223+0000","last_fullsized":"2024-04-24T22:39:31.623223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_clean_scrub_stamp":"2024-04-24T22:34:20.136986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:31.054930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550576+0000","last_change":"2024-04-24T22:39:30.568169+0000","last_active":"2024-04-24T22:39:31.550576+0000","last_peered":"2024-04-24T22:39:31.550576+0000","last_clean":"2024-04-24T22:39:31.550576+0000","last_became_active":"2024-04-24T22:39:30.567221+0000","last_became_peered":"2024-04-24T22:39:30.567221+0000","last_unstale":"2024-04-24T22:39:31.550576+0000","last_undegraded":"2024-04-24T22:39:31.550576+0000","last_fullsized":"2024-04-24T22:39:31.550576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:06.414726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.984607+0000","last_change":"2024-04-24T22:39:29.660378+0000","last_active":"2024-04-24T22:39:34.984607+0000","last_peered":"2024-04-24T22:39:34.984607+0000","last_clean":"2024-04-24T22:39:34.984607+0000","last_became_active":"2024-04-24T22:39:29.660050+0000","last_became_peered":"2024-04-24T22:39:29.660050+0000","last_unstale":"2024-04-24T22:39:34.984607+0000","last_undegraded":"2024-04-24T22:39:34.984607+0000","last_fullsized":"2024-04-24T22:39:34.984607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:06.199365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.977529+0000","last_change":"2024-04-24T22:39:29.674925+0000","last_active":"2024-04-24T22:39:33.977529+0000","last_peered":"2024-04-24T22:39:33.977529+0000","last_clean":"2024-04-24T22:39:33.977529+0000","last_became_active":"2024-04-24T22:39:29.674681+0000","last_became_peered":"2024-04-24T22:39:29.674681+0000","last_unstale":"2024-04-24T22:39:33.977529+0000","last_undegraded":"2024-04-24T22:39:33.977529+0000","last_fullsized":"2024-04-24T22:39:33.977529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:04.295567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551107+0000","last_change":"2024-04-24T22:39:30.581061+0000","last_active":"2024-04-24T22:39:31.551107+0000","last_peered":"2024-04-24T22:39:31.551107+0000","last_clean":"2024-04-24T22:39:31.551107+0000","last_became_active":"2024-04-24T22:39:30.579807+0000","last_became_peered":"2024-04-24T22:39:30.579807+0000","last_unstale":"2024-04-24T22:39:31.551107+0000","last_undegraded":"2024-04-24T22:39:31.551107+0000","last_fullsized":"2024-04-24T22:39:31.551107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:18.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.993660+0000","last_change":"2024-04-24T22:39:29.659948+0000","last_active":"2024-04-24T22:39:34.993660+0000","last_peered":"2024-04-24T22:39:34.993660+0000","last_clean":"2024-04-24T22:39:34.993660+0000","last_became_active":"2024-04-24T22:39:29.659606+0000","last_became_peered":"2024-04-24T22:39:29.659606+0000","last_unstale":"2024-04-24T22:39:34.993660+0000","last_undegraded":"2024-04-24T22:39:34.993660+0000","last_fullsized":"2024-04-24T22:39:34.993660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:32.877670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625120+0000","last_change":"2024-04-24T22:39:30.587740+0000","last_active":"2024-04-24T22:39:31.625120+0000","last_peered":"2024-04-24T22:39:31.625120+0000","last_clean":"2024-04-24T22:39:31.625120+0000","last_became_active":"2024-04-24T22:39:30.587063+0000","last_became_peered":"2024-04-24T22:39:30.587063+0000","last_unstale":"2024-04-24T22:39:31.625120+0000","last_undegraded":"2024-04-24T22:39:31.625120+0000","last_fullsized":"2024-04-24T22:39:31.625120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:02.704753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975036+0000","last_change":"2024-04-24T22:39:29.672345+0000","last_active":"2024-04-24T22:39:33.975036+0000","last_peered":"2024-04-24T22:39:33.975036+0000","last_clean":"2024-04-24T22:39:33.975036+0000","last_became_active":"2024-04-24T22:39:29.671957+0000","last_became_peered":"2024-04-24T22:39:29.671957+0000","last_unstale":"2024-04-24T22:39:33.975036+0000","last_undegraded":"2024-04-24T22:39:33.975036+0000","last_fullsized":"2024-04-24T22:39:33.975036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.181431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625761+0000","last_change":"2024-04-24T22:39:30.595273+0000","last_active":"2024-04-24T22:39:31.625761+0000","last_peered":"2024-04-24T22:39:31.625761+0000","last_clean":"2024-04-24T22:39:31.625761+0000","last_became_active":"2024-04-24T22:39:30.594765+0000","last_became_peered":"2024-04-24T22:39:30.594765+0000","last_unstale":"2024-04-24T22:39:31.625761+0000","last_undegraded":"2024-04-24T22:39:31.625761+0000","last_fullsized":"2024-04-24T22:39:31.625761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.836241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975351+0000","last_change":"2024-04-24T22:39:29.663779+0000","last_active":"2024-04-24T22:39:33.975351+0000","last_peered":"2024-04-24T22:39:33.975351+0000","last_clean":"2024-04-24T22:39:33.975351+0000","last_became_active":"2024-04-24T22:39:29.663484+0000","last_became_peered":"2024-04-24T22:39:29.663484+0000","last_unstale":"2024-04-24T22:39:33.975351+0000","last_undegraded":"2024-04-24T22:39:33.975351+0000","last_fullsized":"2024-04-24T22:39:33.975351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:37.322047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623365+0000","last_change":"2024-04-24T22:39:30.568608+0000","last_active":"2024-04-24T22:39:31.623365+0000","last_peered":"2024-04-24T22:39:31.623365+0000","last_clean":"2024-04-24T22:39:31.623365+0000","last_became_active":"2024-04-24T22:39:30.568271+0000","last_became_peered":"2024-04-24T22:39:30.568271+0000","last_unstale":"2024-04-24T22:39:31.623365+0000","last_undegraded":"2024-04-24T22:39:31.623365+0000","last_fullsized":"2024-04-24T22:39:31.623365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:31.433923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.977036+0000","last_change":"2024-04-24T22:39:29.663745+0000","last_active":"2024-04-24T22:39:33.977036+0000","last_peered":"2024-04-24T22:39:33.977036+0000","last_clean":"2024-04-24T22:39:33.977036+0000","last_became_active":"2024-04-24T22:39:29.663436+0000","last_became_peered":"2024-04-24T22:39:29.663436+0000","last_unstale":"2024-04-24T22:39:33.977036+0000","last_undegraded":"2024-04-24T22:39:33.977036+0000","last_fullsized":"2024-04-24T22:39:33.977036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.502177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992744+0000","last_change":"2024-04-24T22:39:30.580645+0000","last_active":"2024-04-24T22:39:31.992744+0000","last_peered":"2024-04-24T22:39:31.992744+0000","last_clean":"2024-04-24T22:39:31.992744+0000","last_became_active":"2024-04-24T22:39:30.580288+0000","last_became_peered":"2024-04-24T22:39:30.580288+0000","last_unstale":"2024-04-24T22:39:31.992744+0000","last_undegraded":"2024-04-24T22:39:31.992744+0000","last_fullsized":"2024-04-24T22:39:31.992744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:14.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.644717+0000","last_change":"2024-04-24T22:39:29.657571+0000","last_active":"2024-04-24T22:39:33.644717+0000","last_peered":"2024-04-24T22:39:33.644717+0000","last_clean":"2024-04-24T22:39:33.644717+0000","last_became_active":"2024-04-24T22:39:29.657230+0000","last_became_peered":"2024-04-24T22:39:29.657230+0000","last_unstale":"2024-04-24T22:39:33.644717+0000","last_undegraded":"2024-04-24T22:39:33.644717+0000","last_fullsized":"2024-04-24T22:39:33.644717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:26.638528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625230+0000","last_change":"2024-04-24T22:39:30.590468+0000","last_active":"2024-04-24T22:39:31.625230+0000","last_peered":"2024-04-24T22:39:31.625230+0000","last_clean":"2024-04-24T22:39:31.625230+0000","last_became_active":"2024-04-24T22:39:30.590157+0000","last_became_peered":"2024-04-24T22:39:30.590157+0000","last_unstale":"2024-04-24T22:39:31.625230+0000","last_undegraded":"2024-04-24T22:39:31.625230+0000","last_fullsized":"2024-04-24T22:39:31.625230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:33.975204+0000","last_change":"2024-04-24T22:39:29.656600+0000","last_active":"2024-04-24T22:39:33.975204+0000","last_peered":"2024-04-24T22:39:33.975204+0000","last_clean":"2024-04-24T22:39:33.975204+0000","last_became_active":"2024-04-24T22:39:29.656286+0000","last_became_peered":"2024-04-24T22:39:29.656286+0000","last_unstale":"2024-04-24T22:39:33.975204+0000","last_undegraded":"2024-04-24T22:39:33.975204+0000","last_fullsized":"2024-04-24T22:39:33.975204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:33.605969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623587+0000","last_change":"2024-04-24T22:39:30.577213+0000","last_active":"2024-04-24T22:39:31.623587+0000","last_peered":"2024-04-24T22:39:31.623587+0000","last_clean":"2024-04-24T22:39:31.623587+0000","last_became_active":"2024-04-24T22:39:30.576883+0000","last_became_peered":"2024-04-24T22:39:30.576883+0000","last_unstale":"2024-04-24T22:39:31.623587+0000","last_undegraded":"2024-04-24T22:39:31.623587+0000","last_fullsized":"2024-04-24T22:39:31.623587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:16.984591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:34.991575+0000","last_change":"2024-04-24T22:39:29.667082+0000","last_active":"2024-04-24T22:39:34.991575+0000","last_peered":"2024-04-24T22:39:34.991575+0000","last_clean":"2024-04-24T22:39:34.991575+0000","last_became_active":"2024-04-24T22:39:29.666631+0000","last_became_peered":"2024-04-24T22:39:29.666631+0000","last_unstale":"2024-04-24T22:39:34.991575+0000","last_undegraded":"2024-04-24T22:39:34.991575+0000","last_fullsized":"2024-04-24T22:39:34.991575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.001201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624978+0000","last_change":"2024-04-24T22:39:30.565848+0000","last_active":"2024-04-24T22:39:31.624978+0000","last_peered":"2024-04-24T22:39:31.624978+0000","last_clean":"2024-04-24T22:39:31.624978+0000","last_became_active":"2024-04-24T22:39:30.565533+0000","last_became_peered":"2024-04-24T22:39:30.565533+0000","last_unstale":"2024-04-24T22:39:31.624978+0000","last_undegraded":"2024-04-24T22:39:31.624978+0000","last_fullsized":"2024-04-24T22:39:31.624978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:02.406509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623407+0000","last_change":"2024-04-24T22:39:29.662575+0000","last_active":"2024-04-24T22:39:31.623407+0000","last_peered":"2024-04-24T22:39:31.623407+0000","last_clean":"2024-04-24T22:39:31.623407+0000","last_became_active":"2024-04-24T22:39:29.662273+0000","last_became_peered":"2024-04-24T22:39:29.662273+0000","last_unstale":"2024-04-24T22:39:31.623407+0000","last_undegraded":"2024-04-24T22:39:31.623407+0000","last_fullsized":"2024-04-24T22:39:31.623407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:42.661149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623689+0000","last_change":"2024-04-24T22:39:30.587813+0000","last_active":"2024-04-24T22:39:31.623689+0000","last_peered":"2024-04-24T22:39:31.623689+0000","last_clean":"2024-04-24T22:39:31.623689+0000","last_became_active":"2024-04-24T22:39:30.587533+0000","last_became_peered":"2024-04-24T22:39:30.587533+0000","last_unstale":"2024-04-24T22:39:31.623689+0000","last_undegraded":"2024-04-24T22:39:31.623689+0000","last_fullsized":"2024-04-24T22:39:31.623689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:27.042695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551034+0000","last_change":"2024-04-24T22:39:29.664843+0000","last_active":"2024-04-24T22:39:31.551034+0000","last_peered":"2024-04-24T22:39:31.551034+0000","last_clean":"2024-04-24T22:39:31.551034+0000","last_became_active":"2024-04-24T22:39:29.664542+0000","last_became_peered":"2024-04-24T22:39:29.664542+0000","last_unstale":"2024-04-24T22:39:31.551034+0000","last_undegraded":"2024-04-24T22:39:31.551034+0000","last_fullsized":"2024-04-24T22:39:31.551034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:17.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624484+0000","last_change":"2024-04-24T22:39:30.569753+0000","last_active":"2024-04-24T22:39:31.624484+0000","last_peered":"2024-04-24T22:39:31.624484+0000","last_clean":"2024-04-24T22:39:31.624484+0000","last_became_active":"2024-04-24T22:39:30.569433+0000","last_became_peered":"2024-04-24T22:39:30.569433+0000","last_unstale":"2024-04-24T22:39:31.624484+0000","last_undegraded":"2024-04-24T22:39:31.624484+0000","last_fullsized":"2024-04-24T22:39:31.624484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992834+0000","last_change":"2024-04-24T22:39:30.574076+0000","last_active":"2024-04-24T22:39:31.992834+0000","last_peered":"2024-04-24T22:39:31.992834+0000","last_clean":"2024-04-24T22:39:31.992834+0000","last_became_active":"2024-04-24T22:39:30.573769+0000","last_became_peered":"2024-04-24T22:39:30.573769+0000","last_unstale":"2024-04-24T22:39:31.992834+0000","last_undegraded":"2024-04-24T22:39:31.992834+0000","last_fullsized":"2024-04-24T22:39:31.992834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:55.456402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.552897+0000","last_change":"2024-04-24T22:39:30.570762+0000","last_active":"2024-04-24T22:39:31.552897+0000","last_peered":"2024-04-24T22:39:31.552897+0000","last_clean":"2024-04-24T22:39:31.552897+0000","last_became_active":"2024-04-24T22:39:30.570459+0000","last_became_peered":"2024-04-24T22:39:30.570459+0000","last_unstale":"2024-04-24T22:39:31.552897+0000","last_undegraded":"2024-04-24T22:39:31.552897+0000","last_fullsized":"2024-04-24T22:39:31.552897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.826752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.552910+0000","last_change":"2024-04-24T22:39:30.591414+0000","last_active":"2024-04-24T22:39:31.552910+0000","last_peered":"2024-04-24T22:39:31.552910+0000","last_clean":"2024-04-24T22:39:31.552910+0000","last_became_active":"2024-04-24T22:39:30.591135+0000","last_became_peered":"2024-04-24T22:39:30.591135+0000","last_unstale":"2024-04-24T22:39:31.552910+0000","last_undegraded":"2024-04-24T22:39:31.552910+0000","last_fullsized":"2024-04-24T22:39:31.552910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:58.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.625095+0000","last_change":"2024-04-24T22:39:30.583821+0000","last_active":"2024-04-24T22:39:31.625095+0000","last_peered":"2024-04-24T22:39:31.625095+0000","last_clean":"2024-04-24T22:39:31.625095+0000","last_became_active":"2024-04-24T22:39:30.583476+0000","last_became_peered":"2024-04-24T22:39:30.583476+0000","last_unstale":"2024-04-24T22:39:31.625095+0000","last_undegraded":"2024-04-24T22:39:31.625095+0000","last_fullsized":"2024-04-24T22:39:31.625095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:08.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550780+0000","last_change":"2024-04-24T22:39:30.583955+0000","last_active":"2024-04-24T22:39:31.550780+0000","last_peered":"2024-04-24T22:39:31.550780+0000","last_clean":"2024-04-24T22:39:31.550780+0000","last_became_active":"2024-04-24T22:39:30.583632+0000","last_became_peered":"2024-04-24T22:39:30.583632+0000","last_unstale":"2024-04-24T22:39:31.550780+0000","last_undegraded":"2024-04-24T22:39:31.550780+0000","last_fullsized":"2024-04-24T22:39:31.550780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:44.150140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992910+0000","last_change":"2024-04-24T22:39:30.569989+0000","last_active":"2024-04-24T22:39:31.992910+0000","last_peered":"2024-04-24T22:39:31.992910+0000","last_clean":"2024-04-24T22:39:31.992910+0000","last_became_active":"2024-04-24T22:39:30.569680+0000","last_became_peered":"2024-04-24T22:39:30.569680+0000","last_unstale":"2024-04-24T22:39:31.992910+0000","last_undegraded":"2024-04-24T22:39:31.992910+0000","last_fullsized":"2024-04-24T22:39:31.992910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:55.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.553304+0000","last_change":"2024-04-24T22:39:30.572776+0000","last_active":"2024-04-24T22:39:31.553304+0000","last_peered":"2024-04-24T22:39:31.553304+0000","last_clean":"2024-04-24T22:39:31.553304+0000","last_became_active":"2024-04-24T22:39:30.572445+0000","last_became_peered":"2024-04-24T22:39:30.572445+0000","last_unstale":"2024-04-24T22:39:31.553304+0000","last_undegraded":"2024-04-24T22:39:31.553304+0000","last_fullsized":"2024-04-24T22:39:31.553304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:40.058377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624566+0000","last_change":"2024-04-24T22:39:30.587453+0000","last_active":"2024-04-24T22:39:31.624566+0000","last_peered":"2024-04-24T22:39:31.624566+0000","last_clean":"2024-04-24T22:39:31.624566+0000","last_became_active":"2024-04-24T22:39:30.586612+0000","last_became_peered":"2024-04-24T22:39:30.586612+0000","last_unstale":"2024-04-24T22:39:31.624566+0000","last_undegraded":"2024-04-24T22:39:31.624566+0000","last_fullsized":"2024-04-24T22:39:31.624566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:24.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.623057+0000","last_change":"2024-04-24T22:39:30.592921+0000","last_active":"2024-04-24T22:39:31.623057+0000","last_peered":"2024-04-24T22:39:31.623057+0000","last_clean":"2024-04-24T22:39:31.623057+0000","last_became_active":"2024-04-24T22:39:30.592607+0000","last_became_peered":"2024-04-24T22:39:30.592607+0000","last_unstale":"2024-04-24T22:39:31.623057+0000","last_undegraded":"2024-04-24T22:39:31.623057+0000","last_fullsized":"2024-04-24T22:39:31.623057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:11.240743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.624297+0000","last_change":"2024-04-24T22:39:30.586688+0000","last_active":"2024-04-24T22:39:31.624297+0000","last_peered":"2024-04-24T22:39:31.624297+0000","last_clean":"2024-04-24T22:39:31.624297+0000","last_became_active":"2024-04-24T22:39:30.586259+0000","last_became_peered":"2024-04-24T22:39:30.586259+0000","last_unstale":"2024-04-24T22:39:31.624297+0000","last_undegraded":"2024-04-24T22:39:31.624297+0000","last_fullsized":"2024-04-24T22:39:31.624297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:24.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.550809+0000","last_change":"2024-04-24T22:39:30.591940+0000","last_active":"2024-04-24T22:39:31.550809+0000","last_peered":"2024-04-24T22:39:31.550809+0000","last_clean":"2024-04-24T22:39:31.550809+0000","last_became_active":"2024-04-24T22:39:30.591635+0000","last_became_peered":"2024-04-24T22:39:30.591635+0000","last_unstale":"2024-04-24T22:39:31.550809+0000","last_undegraded":"2024-04-24T22:39:31.550809+0000","last_fullsized":"2024-04-24T22:39:31.550809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:16.097707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.551166+0000","last_change":"2024-04-24T22:39:30.588188+0000","last_active":"2024-04-24T22:39:31.551166+0000","last_peered":"2024-04-24T22:39:31.551166+0000","last_clean":"2024-04-24T22:39:31.551166+0000","last_became_active":"2024-04-24T22:39:30.587608+0000","last_became_peered":"2024-04-24T22:39:30.587608+0000","last_unstale":"2024-04-24T22:39:31.551166+0000","last_undegraded":"2024-04-24T22:39:31.551166+0000","last_fullsized":"2024-04-24T22:39:31.551166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:17.690009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T22:39:31.992841+0000","last_change":"2024-04-24T22:39:30.567467+0000","last_active":"2024-04-24T22:39:31.992841+0000","last_peered":"2024-04-24T22:39:31.992841+0000","last_clean":"2024-04-24T22:39:31.992841+0000","last_became_active":"2024-04-24T22:39:30.567162+0000","last_became_peered":"2024-04-24T22:39:30.567162+0000","last_unstale":"2024-04-24T22:39:31.992841+0000","last_undegraded":"2024-04-24T22:39:31.992841+0000","last_fullsized":"2024-04-24T22:39:31.992841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:25.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":125,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":89610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1032079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1523712,"data_stored":1030681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":438728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":459416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":460766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1030365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":39543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:40.366 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T22:39:40.367 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T22:39:40.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T22:39:40.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:40 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1060525163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:40.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:40 smithi171 ceph-mon[32337]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-04-24T22:39:40.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:40 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1060525163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:40.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:40 smithi186 ceph-mon[32753]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-04-24T22:39:40.869 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:41.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:41 smithi171 ceph-mon[32337]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:41.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:41 smithi171 ceph-mon[32337]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:41.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:41 smithi111 ceph-mon[27746]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:41.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:41 smithi111 ceph-mon[27746]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:41.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:41 smithi186 ceph-mon[32753]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:41.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:41 smithi186 ceph-mon[32753]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:39:42.056 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:42 smithi111 ceph-mon[27746]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-24T22:39:42.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:42 smithi171 ceph-mon[32337]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-24T22:39:42.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:42 smithi186 ceph-mon[32753]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-24T22:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 -- 172.21.15.111:0/1950300005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec103e50 msgr2=0x7f61ec10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 --2- 172.21.15.111:0/1950300005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec103e50 0x7f61ec10a700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61e40099c0 tx=0x7f61e4009cd0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 -- 172.21.15.111:0/1950300005 shutdown_connections 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 --2- 172.21.15.111:0/1950300005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec103e50 0x7f61ec10a700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 --2- 172.21.15.111:0/1950300005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec1022b0 0x7f61ec1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 --2- 172.21.15.111:0/1950300005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1034b0 0x7f61ec103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 -- 172.21.15.111:0/1950300005 >> 172.21.15.111:0/1950300005 conn(0x7f61ec0fd850 msgr2=0x7f61ec0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 -- 172.21.15.111:0/1950300005 shutdown_connections 2024-04-24T22:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.187+0000 7f61f2400700 1 -- 172.21.15.111:0/1950300005 wait complete. 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 Processor -- start 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 -- start start 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 0x7f61ec070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec103e50 0x7f61ec1a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10df30 con 0x7f61ec1034b0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10ddb0 con 0x7f61ec103e50 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.189+0000 7f61f2400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10e0b0 con 0x7f61ec1022b0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47850/0 (socket says 172.21.15.111:47850) 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 -- 172.21.15.111:0/785469341 learned_addr learned my addr 172.21.15.111:0/785469341 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eb400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 0x7f61ec070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61ebe00700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec103e50 0x7f61ec1a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 -- 172.21.15.111:0/785469341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 msgr2=0x7f61ec070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 0x7f61ec070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 -- 172.21.15.111:0/785469341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec103e50 msgr2=0x7f61ec1a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec103e50 0x7f61ec1a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eaa00700 1 -- 172.21.15.111:0/785469341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec074170 con 0x7f61ec1034b0 2024-04-24T22:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.190+0000 7f61eb400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 0x7f61ec070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61eaa00700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e9c0 tx=0x7f61d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d800cbd0 con 0x7f61ec1034b0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec1a8b80 con 0x7f61ec1034b0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d800ee60 con 0x7f61ec1034b0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8005630 con 0x7f61ec1034b0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.191+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a9060 con 0x7f61ec1034b0 2024-04-24T22:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.192+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec0507a0 con 0x7f61ec1034b0 2024-04-24T22:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.195+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f61d8004750 con 0x7f61ec1034b0 2024-04-24T22:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.196+0000 7f61e3e00700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 0x7f61d00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.197+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f61d80cf2a0 con 0x7f61ec1034b0 2024-04-24T22:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.197+0000 7f61eb400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 0x7f61d00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.198+0000 7f61eb400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 0x7f61d00a52a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f61ec103310 tx=0x7f61dc00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.201+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f61d809c150 con 0x7f61ec1034b0 2024-04-24T22:39:43.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.512+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f61ec02dd40 con 0x7f61ec1034b0 2024-04-24T22:39:43.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.517+0000 7f61e3e00700 1 -- 172.21.15.111:0/785469341 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f61d809ba50 con 0x7f61ec1034b0 2024-04-24T22:39:43.519 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-04-24T22:39:43.519 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAviilmxaGqHhAATuodqKL4tM1BCuKPVMY8xA== 2024-04-24T22:39:43.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.521+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 msgr2=0x7f61d00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.521+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 0x7f61d00a52a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f61ec103310 tx=0x7f61dc00d040 comp rx=0 tx=0).stop 2024-04-24T22:39:43.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 msgr2=0x7f61ec070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e9c0 tx=0x7f61d800ecd0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 shutdown_connections 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f61d00a2de0 0x7f61d00a52a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61ec103e50 0x7f61ec1a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec1022b0 0x7f61ec070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 --2- 172.21.15.111:0/785469341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec1034b0 0x7f61ec070f50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 >> 172.21.15.111:0/785469341 conn(0x7f61ec0fd850 msgr2=0x7f61ec1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 shutdown_connections 2024-04-24T22:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:43.522+0000 7f61f2400700 1 -- 172.21.15.111:0/785469341 wait complete. 2024-04-24T22:39:43.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:39:43.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:39:43.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:39:43.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:39:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:39:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/785469341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:39:44.375 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:39:44.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T22:39:44.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T22:39:44.495 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T22:39:44.495 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T22:39:44.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T22:39:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:44 smithi171 ceph-mon[32337]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-24T22:39:44.980 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:44 smithi186 ceph-mon[32753]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-24T22:39:45.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:44 smithi111 ceph-mon[27746]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-24T22:39:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:46 smithi171 ceph-mon[32337]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T22:39:46.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:46 smithi111 ceph-mon[27746]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T22:39:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:46 smithi186 ceph-mon[32753]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T22:39:47.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 -- 172.21.15.111:0/3970045009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 msgr2=0x7f21e806ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 --2- 172.21.15.111:0/3970045009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e806ac30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009990 tx=0x7f21d4009ca0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 -- 172.21.15.111:0/3970045009 shutdown_connections 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 --2- 172.21.15.111:0/3970045009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 --2- 172.21.15.111:0/3970045009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 0x7f21e8108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 --2- 172.21.15.111:0/3970045009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e806ac30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.236+0000 7f21ee400700 1 -- 172.21.15.111:0/3970045009 >> 172.21.15.111:0/3970045009 conn(0x7f21e80780a0 msgr2=0x7f21e80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.237+0000 7f21ee400700 1 -- 172.21.15.111:0/3970045009 shutdown_connections 2024-04-24T22:39:47.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.237+0000 7f21ee400700 1 -- 172.21.15.111:0/3970045009 wait complete. 2024-04-24T22:39:47.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 Processor -- start 2024-04-24T22:39:47.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 -- start start 2024-04-24T22:39:47.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e81a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 0x7f21e819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f21e8112550 con 0x7f21e806a7d0 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81123d0 con 0x7f21e81082a0 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.238+0000 7f21ee400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81126d0 con 0x7f21e806b170 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e81a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58362/0 (socket says 172.21.15.111:58362) 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e81a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e7400700 1 -- 172.21.15.111:0/2118408607 learned_addr learned my addr 172.21.15.111:0/2118408607 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:47.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e7e00700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 0x7f21e819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 -- 172.21.15.111:0/2118408607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 msgr2=0x7f21e819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 0x7f21e819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 -- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 msgr2=0x7f21e81a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e81a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 -- 172.21.15.111:0/2118408607 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e819b1a0 con 0x7f21e806b170 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.239+0000 7f21e6a00700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e81a0dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.240+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4026070 con 0x7f21e806b170 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.240+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e819b430 con 0x7f21e806b170 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.240+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d4003d70 con 0x7f21e806b170 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.240+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400fdc0 con 0x7f21e806b170 2024-04-24T22:39:47.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.240+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e81a7750 con 0x7f21e806b170 2024-04-24T22:39:47.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.241+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c8005330 con 0x7f21e806b170 2024-04-24T22:39:47.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.243+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f21d4029070 con 0x7f21e806b170 2024-04-24T22:39:47.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.244+0000 7f21dfe00700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 0x7f21cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.244+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f21d40d0130 con 0x7f21e806b170 2024-04-24T22:39:47.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.245+0000 7f21e7400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 0x7f21cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.245+0000 7f21e7400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 0x7f21cc0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f21e819bb80 tx=0x7f21d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.246+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f21d4030020 con 0x7f21e806b170 2024-04-24T22:39:47.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.534+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f21c80058f0 con 0x7f21e806b170 2024-04-24T22:39:47.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.541+0000 7f21dfe00700 1 -- 172.21.15.111:0/2118408607 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7f21d4013070 con 0x7f21e806b170 2024-04-24T22:39:47.548 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:47.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 msgr2=0x7f21cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 0x7f21cc0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f21e819bb80 tx=0x7f21d8009260 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 msgr2=0x7f21e81a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e81a0dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d400bab0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 shutdown_connections 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f21cc0a2cb0 0x7f21cc0a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21e81082a0 0x7f21e819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21e806a7d0 0x7f21e81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 --2- 172.21.15.111:0/2118408607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21e806b170 0x7f21e81a0dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.546+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 >> 172.21.15.111:0/2118408607 conn(0x7f21e80780a0 msgr2=0x7f21e810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.547+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 shutdown_connections 2024-04-24T22:39:47.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:47.547+0000 7f21ee400700 1 -- 172.21.15.111:0/2118408607 wait complete. 2024-04-24T22:39:47.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2118408607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T22:39:47.944 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2118408607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T22:39:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2118408607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T22:39:48.482 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1872264645},{"type":"v1","addr":"172.21.15.111:6801","nonce":1872264645}]},"active_addr":"172.21.15.111:6801/1872264645","active_change":"2024-04-24T22:31:27.425433+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.111:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":2723701565}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":2298793350}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":564429901}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":1711063394}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3480087064}]}]} 2024-04-24T22:39:48.487 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T22:39:48.487 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T22:39:48.487 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:39:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:48 smithi171 ceph-mon[32337]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-24T22:39:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:48 smithi186 ceph-mon[32753]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-24T22:39:49.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:48 smithi111 ceph-mon[27746]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-24T22:39:50.731 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:50 smithi111 ceph-mon[27746]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-24T22:39:50.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:50 smithi171 ceph-mon[32337]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-24T22:39:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:50 smithi186 ceph-mon[32753]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-24T22:39:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.303+0000 7f6f47000700 1 -- 172.21.15.111:0/2142131311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 msgr2=0x7f6f40107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.303+0000 7f6f47000700 1 --2- 172.21.15.111:0/2142131311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f40107580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b0a0 tx=0x7f6f3000b3b0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 -- 172.21.15.111:0/2142131311 shutdown_connections 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 --2- 172.21.15.111:0/2142131311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101af0 0x7f6f4010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 --2- 172.21.15.111:0/2142131311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f40107580 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 --2- 172.21.15.111:0/2142131311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101130 0x7f6f401015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 -- 172.21.15.111:0/2142131311 >> 172.21.15.111:0/2142131311 conn(0x7f6f400780e0 msgr2=0x7f6f400784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 -- 172.21.15.111:0/2142131311 shutdown_connections 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.304+0000 7f6f47000700 1 -- 172.21.15.111:0/2142131311 wait complete. 2024-04-24T22:39:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 Processor -- start 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 -- start start 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 0x7f6f4019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101af0 0x7f6f4019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40112550 con 0x7f6f401071a0 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401123d0 con 0x7f6f40101130 2024-04-24T22:39:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.305+0000 7f6f47000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401126d0 con 0x7f6f40101af0 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f45000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 0x7f6f4019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f45000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58382/0 (socket says 172.21.15.111:58382) 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f45000700 1 -- 172.21.15.111:0/210762837 learned_addr learned my addr 172.21.15.111:0/210762837 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f37400700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101af0 0x7f6f4019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 -- 172.21.15.111:0/210762837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101af0 msgr2=0x7f6f4019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101af0 0x7f6f4019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 -- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 msgr2=0x7f6f401987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 -- 172.21.15.111:0/210762837 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f40198f90 con 0x7f6f40101130 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f45000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.306+0000 7f6f3fe00700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 0x7f6f4019e630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b070 tx=0x7f6f30004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:51.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.307+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30021070 con 0x7f6f40101130 2024-04-24T22:39:51.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.307+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f30007b80 con 0x7f6f40101130 2024-04-24T22:39:51.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.307+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f40199220 con 0x7f6f40101130 2024-04-24T22:39:51.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.307+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30005040 con 0x7f6f40101130 2024-04-24T22:39:51.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.307+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4010e9a0 con 0x7f6f40101130 2024-04-24T22:39:51.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.308+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f40102850 con 0x7f6f40101130 2024-04-24T22:39:51.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.310+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6f30024070 con 0x7f6f40101130 2024-04-24T22:39:51.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.311+0000 7f6f3d600700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 0x7f6f240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:51.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.312+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6f3000f020 con 0x7f6f40101130 2024-04-24T22:39:51.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.312+0000 7f6f37400700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 0x7f6f240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:51.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.312+0000 7f6f37400700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 0x7f6f240a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f40078750 tx=0x7f6f28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:51.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.314+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f3002b020 con 0x7f6f40101130 2024-04-24T22:39:51.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.539+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f4002db30 con 0x7f6f40101130 2024-04-24T22:39:51.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.541+0000 7f6f3d600700 1 -- 172.21.15.111:0/210762837 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7f6f3009d810 con 0x7f6f40101130 2024-04-24T22:39:51.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:51.544 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":85,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:39:31.546250+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:51.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.544+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 msgr2=0x7f6f240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:51.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.544+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 0x7f6f240a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f40078750 tx=0x7f6f28009260 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.544+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 msgr2=0x7f6f4019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 0x7f6f4019e630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b070 tx=0x7f6f30004d00 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 shutdown_connections 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f401071a0 0x7f6f401987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6f240a2cb0 0x7f6f240a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f40101130 0x7f6f4019e630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 --2- 172.21.15.111:0/210762837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f40101af0 0x7f6f4019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 >> 172.21.15.111:0/210762837 conn(0x7f6f400780e0 msgr2=0x7f6f400ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 shutdown_connections 2024-04-24T22:39:51.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:51.545+0000 7f6f47000700 1 -- 172.21.15.111:0/210762837 wait complete. 2024-04-24T22:39:51.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:51 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/210762837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:51.987 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:51 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/210762837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:51 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/210762837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:52.494 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T22:39:52.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:39:52.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:52 smithi111 ceph-mon[27746]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-24T22:39:52.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:52 smithi171 ceph-mon[32337]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-24T22:39:52.954 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:52 smithi186 ceph-mon[32753]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-24T22:39:54.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:54 smithi111 ceph-mon[27746]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-24T22:39:54.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:54 smithi171 ceph-mon[32337]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-24T22:39:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:54 smithi186 ceph-mon[32753]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-24T22:39:55.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 -- 172.21.15.111:0/1040607713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 msgr2=0x7fdf5c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 --2- 172.21.15.111:0/1040607713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c1015b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009990 tx=0x7fdf48009ca0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 -- 172.21.15.111:0/1040607713 shutdown_connections 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 --2- 172.21.15.111:0/1040607713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 --2- 172.21.15.111:0/1040607713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 0x7fdf5c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 --2- 172.21.15.111:0/1040607713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c1015b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 -- 172.21.15.111:0/1040607713 >> 172.21.15.111:0/1040607713 conn(0x7fdf5c0780e0 msgr2=0x7fdf5c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 -- 172.21.15.111:0/1040607713 shutdown_connections 2024-04-24T22:39:55.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.334+0000 7fdf62a00700 1 -- 172.21.15.111:0/1040607713 wait complete. 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.335+0000 7fdf62a00700 1 Processor -- start 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.335+0000 7fdf62a00700 1 -- start start 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 0x7fdf5c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c111430 con 0x7fdf5c101130 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c1112b0 con 0x7fdf5c101af0 2024-04-24T22:39:55.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf62a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c1115b0 con 0x7fdf5c1071a0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf5be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf5b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf5be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58606/0 (socket says 172.21.15.111:58606) 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf5be00700 1 -- 172.21.15.111:0/3707350774 learned_addr learned my addr 172.21.15.111:0/3707350774 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.336+0000 7fdf60a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 0x7fdf5c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 -- 172.21.15.111:0/3707350774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 msgr2=0x7fdf5c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 0x7fdf5c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 -- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 msgr2=0x7fdf5c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 -- 172.21.15.111:0/3707350774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c198fb0 con 0x7fdf5c101af0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf5b400700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c19eb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf48006010 tx=0x7fdf4800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:55.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48027070 con 0x7fdf5c101af0 2024-04-24T22:39:55.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf480056d0 con 0x7fdf5c101af0 2024-04-24T22:39:55.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48022cf0 con 0x7fdf5c101af0 2024-04-24T22:39:55.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.337+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c199240 con 0x7fdf5c101af0 2024-04-24T22:39:55.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.338+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a54b0 con 0x7fdf5c101af0 2024-04-24T22:39:55.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.339+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c102850 con 0x7fdf5c101af0 2024-04-24T22:39:55.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.341+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdf4802a070 con 0x7fdf5c101af0 2024-04-24T22:39:55.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.343+0000 7fdf58c00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 0x7fdf400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.343+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdf480d08e0 con 0x7fdf5c101af0 2024-04-24T22:39:55.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.343+0000 7fdf5be00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 0x7fdf400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.344+0000 7fdf5be00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 0x7fdf400a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ce0 tx=0x7fdf4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:55.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.345+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf48031020 con 0x7fdf5c101af0 2024-04-24T22:39:55.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.570+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf5c1a5680 con 0x7fdf5c101af0 2024-04-24T22:39:55.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.573+0000 7fdf58c00700 1 -- 172.21.15.111:0/3707350774 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7fdf48013070 con 0x7fdf5c101af0 2024-04-24T22:39:55.575 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:39:55.575 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":85,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:39:31.546250+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:55.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 msgr2=0x7fdf400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 0x7fdf400a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ce0 tx=0x7fdf4c009260 comp rx=0 tx=0).stop 2024-04-24T22:39:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 msgr2=0x7fdf5c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c19eb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf48006010 tx=0x7fdf4800bab0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 shutdown_connections 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf5c1071a0 0x7fdf5c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5c101130 0x7fdf5c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf400a2d30 0x7fdf400a51f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 --2- 172.21.15.111:0/3707350774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf5c101af0 0x7fdf5c19eb90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 >> 172.21.15.111:0/3707350774 conn(0x7fdf5c0780e0 msgr2=0x7fdf5c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 shutdown_connections 2024-04-24T22:39:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:39:55.576+0000 7fdf62a00700 1 -- 172.21.15.111:0/3707350774 wait complete. 2024-04-24T22:39:55.901 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:55 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3707350774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:55.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:55 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3707350774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:56.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:55 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3707350774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:39:56.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T22:39:56.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T22:39:56.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T22:39:56.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T22:39:56.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T22:39:56.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T22:39:56.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T22:39:56.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T22:39:56.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T22:39:56.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T22:39:56.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T22:39:56.283 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T22:39:56.788 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:56 smithi111 ceph-mon[27746]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-24T22:39:56.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:56 smithi171 ceph-mon[32337]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-24T22:39:56.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:56 smithi186 ceph-mon[32753]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-24T22:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:57.288 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:57.294 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:57.298 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:39:58.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:39:58 smithi171 ceph-mon[32337]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:39:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:39:58 smithi186 ceph-mon[32753]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:39:59.023 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:39:58 smithi111 ceph-mon[27746]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:40:00.333 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.338 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.342 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:00.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:00 smithi171 ceph-mon[32337]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:00.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:00 smithi171 ceph-mon[32337]: overall HEALTH_OK 2024-04-24T22:40:00.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:00 smithi111 ceph-mon[27746]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:00.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:00 smithi111 ceph-mon[27746]: overall HEALTH_OK 2024-04-24T22:40:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:00 smithi186 ceph-mon[32753]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:00 smithi186 ceph-mon[32753]: overall HEALTH_OK 2024-04-24T22:40:02.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:02 smithi171 ceph-mon[32337]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:02.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:02 smithi186 ceph-mon[32753]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:03.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:02 smithi111 ceph-mon[27746]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:04.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:04 smithi111 ceph-mon[27746]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:04.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:04 smithi171 ceph-mon[32337]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:04.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:04 smithi186 ceph-mon[32753]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 -- 172.21.15.111:0/529979126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 msgr2=0x7fcfb006b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/529979126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb006b2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009990 tx=0x7fcf9c009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 -- 172.21.15.111:0/529979126 shutdown_connections 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/529979126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb010eb80 0x7fcfb0110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/529979126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfb0103180 0x7fcfb006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/529979126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb006b2c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 -- 172.21.15.111:0/529979126 >> 172.21.15.111:0/529979126 conn(0x7fcfb00780e0 msgr2=0x7fcfb00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 -- 172.21.15.111:0/529979126 shutdown_connections 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.896+0000 7fcfb4c00700 1 -- 172.21.15.111:0/529979126 wait complete. 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.897+0000 7fcfb4c00700 1 Processor -- start 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.897+0000 7fcfb4c00700 1 -- start start 2024-04-24T22:40:05.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb01a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0103180 0x7fcfb01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfb010eb80 0x7fcfb019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb0114730 con 0x7fcfb006ae40 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01145b0 con 0x7fcfb0103180 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfb4c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01148b0 con 0x7fcfb010eb80 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb01a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb01a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42242/0 (socket says 172.21.15.111:42242) 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 -- 172.21.15.111:0/2396087711 learned_addr learned my addr 172.21.15.111:0/2396087711 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfad400700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0103180 0x7fcfb01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfae800700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfb010eb80 0x7fcfb019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 -- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfb010eb80 msgr2=0x7fcfb019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfb010eb80 0x7fcfb019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 -- 172.21.15.111:0/2396087711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0103180 msgr2=0x7fcfb01a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0103180 0x7fcfb01a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.898+0000 7fcfade00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb019b110 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfade00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb01a07f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000cbd0 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb019b400 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa000ee60 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0005630 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.899+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb01a76c0 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.900+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfb0100f60 con 0x7fcfb006ae40 2024-04-24T22:40:05.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.902+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcfa0004750 con 0x7fcfb006ae40 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfa6a00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcf940a2d30 0x7fcf940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcfa00cf7d0 con 0x7fcfb006ae40 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfad400700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcf940a2d30 0x7fcf940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfa6a00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] conn(0x7fcf940ac510 0x7fcf940ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfa0017e50 con 0x7fcf940ac510 2024-04-24T22:40:05.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.904+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcfa00cfbc0 con 0x7fcfb006ae40 2024-04-24T22:40:05.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.905+0000 7fcfae800700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] conn(0x7fcf940ac510 0x7fcf940ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.905+0000 7fcfad400700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcf940a2d30 0x7fcf940a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b2e0 tx=0x7fcf9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.905+0000 7fcfae800700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] conn(0x7fcf940ac510 0x7fcf940ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.906+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== osd.2 v2:172.21.15.186:6800/2871610305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfa0017e50 con 0x7fcf940ac510 2024-04-24T22:40:05.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.924+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 --> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcfb006a1b0 con 0x7fcf940ac510 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.925+0000 7fcfa6a00700 1 -- 172.21.15.111:0/2396087711 <== osd.2 v2:172.21.15.186:6800/2871610305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcfb006a1b0 con 0x7fcf940ac510 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.925+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] conn(0x7fcf940ac510 msgr2=0x7fcf940ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.925+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] conn(0x7fcf940ac510 0x7fcf940ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.925+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcf940a2d30 msgr2=0x7fcf940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.925+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcf940a2d30 0x7fcf940a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b2e0 tx=0x7fcf9c005960 comp rx=0 tx=0).stop 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 msgr2=0x7fcfb01a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 --2- 172.21.15.111:0/2396087711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfb006ae40 0x7fcfb01a07f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfae800700 1 -- 172.21.15.111:0/2396087711 reap_dead start 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 shutdown_connections 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 >> 172.21.15.111:0/2396087711 conn(0x7fcfb00780e0 msgr2=0x7fcfb0100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 shutdown_connections 2024-04-24T22:40:05.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:04.926+0000 7fcfb4c00700 1 -- 172.21.15.111:0/2396087711 wait complete. 2024-04-24T22:40:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:06 smithi171 ceph-mon[32337]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:40:06.953 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:06 smithi111 ceph-mon[27746]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:40:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:06 smithi186 ceph-mon[32753]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:40:07.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 -- 172.21.15.111:0/3411498770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f151406a8e0 msgr2=0x7f151410f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 --2- 172.21.15.111:0/3411498770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f151406a8e0 0x7f151410f0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f150c00b0a0 tx=0x7f150c00b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 -- 172.21.15.111:0/3411498770 shutdown_connections 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 --2- 172.21.15.111:0/3411498770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f151406a8e0 0x7f151410f0f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 --2- 172.21.15.111:0/3411498770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15141082b0 0x7f1514108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 --2- 172.21.15.111:0/3411498770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1514069f40 0x7f151406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 -- 172.21.15.111:0/3411498770 >> 172.21.15.111:0/3411498770 conn(0x7f15140780e0 msgr2=0x7f15140784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 -- 172.21.15.111:0/3411498770 shutdown_connections 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.926+0000 7f151ae00700 1 -- 172.21.15.111:0/3411498770 wait complete. 2024-04-24T22:40:07.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.928+0000 7f151ae00700 1 Processor -- start 2024-04-24T22:40:07.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.928+0000 7f151ae00700 1 -- start start 2024-04-24T22:40:07.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 0x7f15141a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f151406a8e0 0x7f15141a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15141082b0 0x7f151419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1514112550 con 0x7f151406a8e0 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15141123d0 con 0x7f1514069f40 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f151ae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f15141126d0 con 0x7f15141082b0 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f1513e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 0x7f15141a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f1513e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 0x7f15141a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43460/0 (socket says 172.21.15.111:43460) 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f1513e00700 1 -- 172.21.15.111:0/3532980480 learned_addr learned my addr 172.21.15.111:0/3532980480 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f1513400700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f151406a8e0 0x7f15141a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.929+0000 7f1518e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15141082b0 0x7f151419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 -- 172.21.15.111:0/3532980480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15141082b0 msgr2=0x7f151419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15141082b0 0x7f151419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 -- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f151406a8e0 msgr2=0x7f15141a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f151406a8e0 0x7f15141a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151419b1e0 con 0x7f1514069f40 2024-04-24T22:40:07.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1513e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 0x7f15141a08c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f150400e9a0 tx=0x7f150400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400ca10 con 0x7f1514069f40 2024-04-24T22:40:07.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150400cbb0 con 0x7f1514069f40 2024-04-24T22:40:07.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151419b4d0 con 0x7f1514069f40 2024-04-24T22:40:07.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504018680 con 0x7f1514069f40 2024-04-24T22:40:07.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.930+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a7790 con 0x7f1514069f40 2024-04-24T22:40:07.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.931+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f151410de50 con 0x7f1514069f40 2024-04-24T22:40:07.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.933+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1504010430 con 0x7f1514069f40 2024-04-24T22:40:07.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1510c00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f80a2d30 0x7f14f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f15040cfaa0 con 0x7f1514069f40 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1513400700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f80a2d30 0x7f14f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1510c00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] conn(0x7f14f80ac510 0x7f14f80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1504019ec0 con 0x7f14f80ac510 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.935+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f15040cfe50 con 0x7f1514069f40 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.936+0000 7f1513400700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f80a2d30 0x7f14f80a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15141031c0 tx=0x7f1500009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.936+0000 7f1518e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] conn(0x7f14f80ac510 0x7f14f80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.936+0000 7f1518e00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] conn(0x7f14f80ac510 0x7f14f80ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.937+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== osd.4 v2:172.21.15.171:6808/2305118347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1504019ec0 con 0x7f14f80ac510 2024-04-24T22:40:07.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.955+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 --> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1514103960 con 0x7f14f80ac510 2024-04-24T22:40:07.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f1510c00700 1 -- 172.21.15.111:0/3532980480 <== osd.4 v2:172.21.15.171:6808/2305118347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1514103960 con 0x7f14f80ac510 2024-04-24T22:40:07.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] conn(0x7f14f80ac510 msgr2=0x7f14f80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] conn(0x7f14f80ac510 0x7f14f80ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f80a2d30 msgr2=0x7f14f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f14f80a2d30 0x7f14f80a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15141031c0 tx=0x7f1500009260 comp rx=0 tx=0).stop 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 msgr2=0x7f15141a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f151ae00700 1 --2- 172.21.15.111:0/3532980480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1514069f40 0x7f15141a08c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f150400e9a0 tx=0x7f150400ecb0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.956+0000 7f1518e00700 1 -- 172.21.15.111:0/3532980480 reap_dead start 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.957+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 shutdown_connections 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.957+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 >> 172.21.15.111:0/3532980480 conn(0x7f15140780e0 msgr2=0x7f151410d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.957+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 shutdown_connections 2024-04-24T22:40:07.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.957+0000 7f151ae00700 1 -- 172.21.15.111:0/3532980480 wait complete. 2024-04-24T22:40:08.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:08 smithi111 ceph-mon[27746]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:40:08.807 INFO:teuthology.orchestra.run.smithi111.stdout:77309411401 2024-04-24T22:40:08.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 -- 172.21.15.111:0/53363262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a8102ca0 msgr2=0x7fd6a810f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 --2- 172.21.15.111:0/53363262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a8102ca0 0x7fd6a810f110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b0d0 tx=0x7fd6a000b3e0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 -- 172.21.15.111:0/53363262 shutdown_connections 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 --2- 172.21.15.111:0/53363262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a8102ca0 0x7fd6a810f110 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 --2- 172.21.15.111:0/53363262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81082f0 0x7fd6a81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 --2- 172.21.15.111:0/53363262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a81022e0 0x7fd6a8102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 -- 172.21.15.111:0/53363262 >> 172.21.15.111:0/53363262 conn(0x7fd6a8079390 msgr2=0x7fd6a80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:08.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 -- 172.21.15.111:0/53363262 shutdown_connections 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.814+0000 7fd6ad800700 1 -- 172.21.15.111:0/53363262 wait complete. 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.815+0000 7fd6ad800700 1 Processor -- start 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.815+0000 7fd6ad800700 1 -- start start 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81022e0 0x7fd6a81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a8102ca0 0x7fd6a81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a81082f0 0x7fd6a819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81125c0 con 0x7fd6a81082f0 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112440 con 0x7fd6a8102ca0 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6ad800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112740 con 0x7fd6a81022e0 2024-04-24T22:40:08.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a81082f0 0x7fd6a819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a6000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a8102ca0 0x7fd6a81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81022e0 0x7fd6a81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a6000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a8102ca0 0x7fd6a81a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43472/0 (socket says 172.21.15.111:43472) 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a81082f0 0x7fd6a819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42320/0 (socket says 172.21.15.111:42320) 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.816+0000 7fd6a6000700 1 -- 172.21.15.111:0/426089680 learned_addr learned my addr 172.21.15.111:0/426089680 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 -- 172.21.15.111:0/426089680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a8102ca0 msgr2=0x7fd6a81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6a8102ca0 0x7fd6a81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 -- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a81082f0 msgr2=0x7fd6a819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6a81082f0 0x7fd6a819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a819b0d0 con 0x7fd6a81022e0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6a6a00700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81022e0 0x7fd6a81a07b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd69800e9f0 tx=0x7fd69800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800c9c0 con 0x7fd6a81022e0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a819b3c0 con 0x7fd6a81022e0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd69800cb60 con 0x7fd6a81022e0 2024-04-24T22:40:08.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698018680 con 0x7fd6a81022e0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.817+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a81a7680 con 0x7fd6a81022e0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.818+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6a810eeb0 con 0x7fd6a81022e0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.820+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd698010430 con 0x7fd6a81022e0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd69f400700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd68c0a2d30 0x7fd68c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd6980cf5a0 con 0x7fd6a81022e0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd6a6000700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd68c0a2d30 0x7fd68c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd69f400700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] conn(0x7fd68c0ac570 0x7fd68c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd69801bec0 con 0x7fd68c0ac570 2024-04-24T22:40:08.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.822+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd6980cf950 con 0x7fd6a81022e0 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.823+0000 7fd6a7400700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] conn(0x7fd68c0ac570 0x7fd68c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.823+0000 7fd6a6000700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd68c0a2d30 0x7fd68c0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8079d10 tx=0x7fd694007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.823+0000 7fd6a7400700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] conn(0x7fd68c0ac570 0x7fd68c0ae990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.824+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== osd.6 v2:172.21.15.111:6818/525582463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd69801bec0 con 0x7fd68c0ac570 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.842+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 --> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6a81039a0 con 0x7fd68c0ac570 2024-04-24T22:40:08.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd69f400700 1 -- 172.21.15.111:0/426089680 <== osd.6 v2:172.21.15.111:6818/525582463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6a81039a0 con 0x7fd68c0ac570 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] conn(0x7fd68c0ac570 msgr2=0x7fd68c0ae990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] conn(0x7fd68c0ac570 0x7fd68c0ae990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd68c0a2d30 msgr2=0x7fd68c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd68c0a2d30 0x7fd68c0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8079d10 tx=0x7fd694007480 comp rx=0 tx=0).stop 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81022e0 msgr2=0x7fd6a81a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 --2- 172.21.15.111:0/426089680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6a81022e0 0x7fd6a81a07b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd69800e9f0 tx=0x7fd69800ed00 comp rx=0 tx=0).stop 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6a7400700 1 -- 172.21.15.111:0/426089680 reap_dead start 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 shutdown_connections 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.843+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 >> 172.21.15.111:0/426089680 conn(0x7fd6a8079390 msgr2=0x7fd6a810d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.844+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 shutdown_connections 2024-04-24T22:40:08.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:08.844+0000 7fd6ad800700 1 -- 172.21.15.111:0/426089680 wait complete. 2024-04-24T22:40:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:08 smithi171 ceph-mon[32337]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:40:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:08 smithi186 ceph-mon[32753]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:40:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:10 smithi171 ceph-mon[32337]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T22:40:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:10 smithi186 ceph-mon[32753]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T22:40:11.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:10 smithi111 ceph-mon[27746]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T22:40:11.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 -- 172.21.15.111:0/853641395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 msgr2=0x7f815c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 --2- 172.21.15.111:0/853641395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c10ec40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f814c009990 tx=0x7f814c009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 -- 172.21.15.111:0/853641395 shutdown_connections 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 --2- 172.21.15.111:0/853641395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f815c109220 0x7f815c115730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 --2- 172.21.15.111:0/853641395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c10ec40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 --2- 172.21.15.111:0/853641395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f815c108860 0x7f815c108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 -- 172.21.15.111:0/853641395 >> 172.21.15.111:0/853641395 conn(0x7f815c075a00 msgr2=0x7f815c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 -- 172.21.15.111:0/853641395 shutdown_connections 2024-04-24T22:40:11.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.119+0000 7f8161800700 1 -- 172.21.15.111:0/853641395 wait complete. 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.120+0000 7f8161800700 1 Processor -- start 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 -- start start 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f815c108860 0x7f815c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f815c109220 0x7f815c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c1a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f815c118bb0 con 0x7f815c10e860 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f815c118a30 con 0x7f815c108860 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f8161800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f815c118d30 con 0x7f815c109220 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f815b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c1a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f815b400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c1a3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42274/0 (socket says 172.21.15.111:42274) 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f815b400700 1 -- 172.21.15.111:0/3189278306 learned_addr learned my addr 172.21.15.111:0/3189278306 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f815aa00700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f815c108860 0x7f815c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.121+0000 7f815a000700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f815c109220 0x7f815c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 -- 172.21.15.111:0/3189278306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f815c109220 msgr2=0x7f815c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f815c109220 0x7f815c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 -- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f815c108860 msgr2=0x7f815c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f815c108860 0x7f815c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1a3fb0 con 0x7f815c10e860 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f815b400700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c1a3870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f815400c370 tx=0x7f815400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540092e0 con 0x7f815c10e860 2024-04-24T22:40:11.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8154003a00 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c1a4240 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540047c0 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.122+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1abb20 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.123+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f815c0629c0 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.125+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f815401a020 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f8153400700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f81400a2cb0 0x7f81400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f81540cf3e0 con 0x7f815c10e860 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f815aa00700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f81400a2cb0 0x7f81400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f8153400700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] conn(0x7f81400ac490 0x7f81400ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f815400be50 con 0x7f81400ac490 2024-04-24T22:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.127+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f81540cf790 con 0x7f815c10e860 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.128+0000 7f815a000700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] conn(0x7f81400ac490 0x7f81400ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.128+0000 7f815aa00700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f81400a2cb0 0x7f81400a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.128+0000 7f815a000700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] conn(0x7f81400ac490 0x7f81400ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.129+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== osd.10 v2:172.21.15.171:6824/4154918762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f815400be50 con 0x7f81400ac490 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.147+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 --> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f815c062e40 con 0x7f81400ac490 2024-04-24T22:40:11.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.147+0000 7f8153400700 1 -- 172.21.15.111:0/3189278306 <== osd.10 v2:172.21.15.171:6824/4154918762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f815c062e40 con 0x7f81400ac490 2024-04-24T22:40:11.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] conn(0x7f81400ac490 msgr2=0x7f81400ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] conn(0x7f81400ac490 0x7f81400ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f81400a2cb0 msgr2=0x7f81400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f81400a2cb0 0x7f81400a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c005960 comp rx=0 tx=0).stop 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 msgr2=0x7f815c1a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 --2- 172.21.15.111:0/3189278306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f815c10e860 0x7f815c1a3870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f815400c370 tx=0x7f815400c680 comp rx=0 tx=0).stop 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f815b400700 1 -- 172.21.15.111:0/3189278306 reap_dead start 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 shutdown_connections 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 >> 172.21.15.111:0/3189278306 conn(0x7f815c075a00 msgr2=0x7f815c075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 shutdown_connections 2024-04-24T22:40:11.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:07.148+0000 7f8161800700 1 -- 172.21.15.111:0/3189278306 wait complete. 2024-04-24T22:40:12.776 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:12 smithi111 ceph-mon[27746]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T22:40:12.872 INFO:teuthology.orchestra.run.smithi111.stdout:141733920829 2024-04-24T22:40:12.872 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T22:40:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:12 smithi186 ceph-mon[32753]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T22:40:13.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:12 smithi171 ceph-mon[32337]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T22:40:13.298 INFO:teuthology.orchestra.run.smithi111.stdout:197568495665 2024-04-24T22:40:13.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T22:40:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:14 smithi111 ceph-mon[27746]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:14.918 INFO:teuthology.orchestra.run.smithi111.stdout:309237645338 2024-04-24T22:40:14.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T22:40:14.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:14 smithi186 ceph-mon[32753]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:15.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:14 smithi171 ceph-mon[32337]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:16.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.961+0000 7f671ca00700 1 -- 172.21.15.111:0/665413789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671810ecd0 msgr2=0x7f67181110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.961+0000 7f671ca00700 1 --2- 172.21.15.111:0/665413789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671810ecd0 0x7f67181110c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f671000b0a0 tx=0x7f671000b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 -- 172.21.15.111:0/665413789 shutdown_connections 2024-04-24T22:40:16.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 --2- 172.21.15.111:0/665413789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671810ecd0 0x7f67181110c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 --2- 172.21.15.111:0/665413789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671806a7d0 0x7f671806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 --2- 172.21.15.111:0/665413789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671806b0f0 0x7f671810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 -- 172.21.15.111:0/665413789 >> 172.21.15.111:0/665413789 conn(0x7f67180780a0 msgr2=0x7f67180784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 -- 172.21.15.111:0/665413789 shutdown_connections 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.962+0000 7f671ca00700 1 -- 172.21.15.111:0/665413789 wait complete. 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.963+0000 7f671ca00700 1 Processor -- start 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.963+0000 7f671ca00700 1 -- start start 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 0x7f67181a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671806b0f0 0x7f67181a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671810ecd0 0x7f671819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6718114770 con 0x7f671806a7d0 2024-04-24T22:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f67181145f0 con 0x7f671806b0f0 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f671ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67181148f0 con 0x7f671810ecd0 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f6715e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 0x7f67181a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f6715e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 0x7f67181a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55280/0 (socket says 172.21.15.111:55280) 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f6715e00700 1 -- 172.21.15.111:0/1925715009 learned_addr learned my addr 172.21.15.111:0/1925715009 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f670d400700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671806b0f0 0x7f67181a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f6716800700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671810ecd0 0x7f671819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.964+0000 7f6715e00700 1 -- 172.21.15.111:0/1925715009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671810ecd0 msgr2=0x7f671819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6715e00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671810ecd0 0x7f671819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6715e00700 1 -- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671806b0f0 msgr2=0x7f67181a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6715e00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671806b0f0 0x7f67181a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6715e00700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671819b150 con 0x7f671806a7d0 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6716800700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f671810ecd0 0x7f671819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:16.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f6715e00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 0x7f67181a0830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9c0 tx=0x7f670800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800cbd0 con 0x7f671806a7d0 2024-04-24T22:40:16.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671819b440 con 0x7f671806a7d0 2024-04-24T22:40:16.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f670800ee60 con 0x7f671806a7d0 2024-04-24T22:40:16.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181a7700 con 0x7f671806a7d0 2024-04-24T22:40:16.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.965+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708005650 con 0x7f671806a7d0 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.966+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67180629c0 con 0x7f671806a7d0 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.968+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6708004750 con 0x7f671806a7d0 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.969+0000 7f670f400700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66fc0a2cb0 0x7f66fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f67080cfd80 con 0x7f671806a7d0 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670d400700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66fc0a2cb0 0x7f66fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670f400700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] conn(0x7f66fc0ac490 0x7f66fc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f670801aec0 con 0x7f66fc0ac490 2024-04-24T22:40:16.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f67080d0130 con 0x7f671806a7d0 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f6716800700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] conn(0x7f66fc0ac490 0x7f66fc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.970+0000 7f670d400700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66fc0a2cb0 0x7f66fc0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6700009680 tx=0x7f6700009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.971+0000 7f6716800700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] conn(0x7f66fc0ac490 0x7f66fc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.972+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== osd.1 v2:172.21.15.171:6800/1334857356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f670801aec0 con 0x7f66fc0ac490 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.990+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 --> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6718062dc0 con 0x7f66fc0ac490 2024-04-24T22:40:16.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.990+0000 7f670f400700 1 -- 172.21.15.111:0/1925715009 <== osd.1 v2:172.21.15.171:6800/1334857356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6718062dc0 con 0x7f66fc0ac490 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] conn(0x7f66fc0ac490 msgr2=0x7f66fc0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] conn(0x7f66fc0ac490 0x7f66fc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66fc0a2cb0 msgr2=0x7f66fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66fc0a2cb0 0x7f66fc0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6700009680 tx=0x7f6700009260 comp rx=0 tx=0).stop 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 msgr2=0x7f67181a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 --2- 172.21.15.111:0/1925715009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f671806a7d0 0x7f67181a0830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9c0 tx=0x7f670800ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f6716800700 1 -- 172.21.15.111:0/1925715009 reap_dead start 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 shutdown_connections 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 >> 172.21.15.111:0/1925715009 conn(0x7f67180780a0 msgr2=0x7f6718110080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:16.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 shutdown_connections 2024-04-24T22:40:16.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:15.991+0000 7f671ca00700 1 -- 172.21.15.111:0/1925715009 wait complete. 2024-04-24T22:40:16.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.195+0000 7f2fe3c00700 1 -- 172.21.15.111:0/1632727097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc10e860 msgr2=0x7f2fdc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.195+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/1632727097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc10e860 0x7f2fdc10ec40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009990 tx=0x7f2fd0009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.195+0000 7f2fe3c00700 1 -- 172.21.15.111:0/1632727097 shutdown_connections 2024-04-24T22:40:16.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.195+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/1632727097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc109220 0x7f2fdc115730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.196+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/1632727097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc10e860 0x7f2fdc10ec40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.196+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/1632727097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc108860 0x7f2fdc108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.196+0000 7f2fe3c00700 1 -- 172.21.15.111:0/1632727097 >> 172.21.15.111:0/1632727097 conn(0x7f2fdc075a00 msgr2=0x7f2fdc107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.196+0000 7f2fe3c00700 1 -- 172.21.15.111:0/1632727097 shutdown_connections 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.196+0000 7f2fe3c00700 1 -- 172.21.15.111:0/1632727097 wait complete. 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 Processor -- start 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 -- start start 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 0x7f2fdc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc109220 0x7f2fdc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc10e860 0x7f2fdc1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc118c00 con 0x7f2fdc108860 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc118a80 con 0x7f2fdc109220 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe3c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc118d80 con 0x7f2fdc10e860 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe1200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 0x7f2fdc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe1200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 0x7f2fdc19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55298/0 (socket says 172.21.15.111:55298) 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe1200700 1 -- 172.21.15.111:0/3594964140 learned_addr learned my addr 172.21.15.111:0/3594964140 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.197+0000 7f2fe1c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc10e860 0x7f2fdc1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 -- 172.21.15.111:0/3594964140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc109220 msgr2=0x7f2fdc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fdc109220 0x7f2fdc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 -- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 msgr2=0x7f2fdc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 0x7f2fdc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc1a4030 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1200700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fdc108860 0x7f2fdc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fe1c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc10e860 0x7f2fdc1a38f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400efb0 tx=0x7f2fd400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.198+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ca50 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.199+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc1a4320 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.199+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd400cbf0 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.199+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4007500 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.199+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1abba0 con 0x7f2fdc10e860 2024-04-24T22:40:16.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.199+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fdc0629c0 con 0x7f2fdc10e860 2024-04-24T22:40:16.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.202+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2fd4012050 con 0x7f2fdc10e860 2024-04-24T22:40:16.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.203+0000 7f2fd9600700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2fc40a2d30 0x7f2fc40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.203+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2fd40cee40 con 0x7f2fdc10e860 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.203+0000 7f2fe1200700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2fc40a2d30 0x7f2fc40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.204+0000 7f2fd9600700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] conn(0x7f2fc40ac510 0x7f2fc40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.204+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fc40aefe0 con 0x7f2fc40ac510 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.204+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2fd40cf1f0 con 0x7f2fdc10e860 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.204+0000 7f2fdbe00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] conn(0x7f2fc40ac510 0x7f2fc40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.204+0000 7f2fe1200700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2fc40a2d30 0x7f2fc40a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006010 tx=0x7f2fd000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.205+0000 7f2fdbe00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] conn(0x7f2fc40ac510 0x7f2fc40ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:16.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.206+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== osd.3 v2:172.21.15.111:6810/3102831357 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fc40aefe0 con 0x7f2fc40ac510 2024-04-24T22:40:16.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.223+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 --> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fdc062dc0 con 0x7f2fc40ac510 2024-04-24T22:40:16.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.224+0000 7f2fd9600700 1 -- 172.21.15.111:0/3594964140 <== osd.3 v2:172.21.15.111:6810/3102831357 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fdc062dc0 con 0x7f2fc40ac510 2024-04-24T22:40:16.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.224+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] conn(0x7f2fc40ac510 msgr2=0x7f2fc40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.224+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] conn(0x7f2fc40ac510 0x7f2fc40ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2fc40a2d30 msgr2=0x7f2fc40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2fc40a2d30 0x7f2fc40a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006010 tx=0x7f2fd000b260 comp rx=0 tx=0).stop 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc10e860 msgr2=0x7f2fdc1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 --2- 172.21.15.111:0/3594964140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fdc10e860 0x7f2fdc1a38f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400efb0 tx=0x7f2fd400c3a0 comp rx=0 tx=0).stop 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe1c00700 1 -- 172.21.15.111:0/3594964140 reap_dead start 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 shutdown_connections 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 >> 172.21.15.111:0/3594964140 conn(0x7f2fdc075a00 msgr2=0x7f2fdc075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 shutdown_connections 2024-04-24T22:40:16.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:16.225+0000 7f2fe3c00700 1 -- 172.21.15.111:0/3594964140 wait complete. 2024-04-24T22:40:16.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:16 smithi111 ceph-mon[27746]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:16.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:16 smithi186 ceph-mon[32753]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 -- 172.21.15.111:0/3780150409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c104f60 msgr2=0x7fc54c108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 --2- 172.21.15.111:0/3780150409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c104f60 0x7fc54c108fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc5380099c0 tx=0x7fc538009cd0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 -- 172.21.15.111:0/3780150409 shutdown_connections 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 --2- 172.21.15.111:0/3780150409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c109690 0x7fc54c111260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 --2- 172.21.15.111:0/3780150409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 --2- 172.21.15.111:0/3780150409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c104f60 0x7fc54c108fb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 -- 172.21.15.111:0/3780150409 >> 172.21.15.111:0/3780150409 conn(0x7fc54c0ffe20 msgr2=0x7fc54c102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 -- 172.21.15.111:0/3780150409 shutdown_connections 2024-04-24T22:40:17.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.146+0000 7fc552e00700 1 -- 172.21.15.111:0/3780150409 wait complete. 2024-04-24T22:40:17.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 Processor -- start 2024-04-24T22:40:17.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 -- start start 2024-04-24T22:40:17.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c104f60 0x7fc54c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c109690 0x7fc54c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c1146c0 con 0x7fc54c109690 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c114540 con 0x7fc54c1045b0 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.148+0000 7fc552e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c114840 con 0x7fc54c104f60 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c104f60 0x7fc54c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44562/0 (socket says 172.21.15.111:44562) 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c104f60 0x7fc54c1a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:47472/0 (socket says 172.21.15.111:47472) 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 -- 172.21.15.111:0/1170318393 learned_addr learned my addr 172.21.15.111:0/1170318393 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc552400700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c109690 0x7fc54c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 -- 172.21.15.111:0/1170318393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c104f60 msgr2=0x7fc54c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc54c104f60 0x7fc54c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 -- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c109690 msgr2=0x7fc54c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc54c109690 0x7fc54c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.149+0000 7fc551a00700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c19b100 con 0x7fc54c1045b0 2024-04-24T22:40:17.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc551a00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c1a0790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc54000eb30 tx=0x7fc54000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000cbd0 con 0x7fc54c1045b0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc540004500 con 0x7fc54c1045b0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c19b3f0 con 0x7fc54c1045b0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540010430 con 0x7fc54c1045b0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.150+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c1a7660 con 0x7fc54c1045b0 2024-04-24T22:40:17.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.151+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc54c0629c0 con 0x7fc54c1045b0 2024-04-24T22:40:17.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:16 smithi171 ceph-mon[32337]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:17.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.153+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc540003f00 con 0x7fc54c1045b0 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc54a000700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc5340a2d30 0x7fc5340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc5400cf710 con 0x7fc54c1045b0 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc551000700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc5340a2d30 0x7fc5340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc54a000700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] conn(0x7fc5340ac510 0x7fc5340ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc540023e50 con 0x7fc5340ac510 2024-04-24T22:40:17.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fc5400cfb00 con 0x7fc54c1045b0 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.155+0000 7fc552400700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] conn(0x7fc5340ac510 0x7fc5340ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.156+0000 7fc551000700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc5340a2d30 0x7fc5340a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.156+0000 7fc552400700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] conn(0x7fc5340ac510 0x7fc5340ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:17.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.157+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== osd.7 v2:172.21.15.171:6816/2409942140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc540023e50 con 0x7fc5340ac510 2024-04-24T22:40:17.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.175+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 --> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc54c062e40 con 0x7fc5340ac510 2024-04-24T22:40:17.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.176+0000 7fc54a000700 1 -- 172.21.15.111:0/1170318393 <== osd.7 v2:172.21.15.171:6816/2409942140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc54c062e40 con 0x7fc5340ac510 2024-04-24T22:40:17.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] conn(0x7fc5340ac510 msgr2=0x7fc5340ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] conn(0x7fc5340ac510 0x7fc5340ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc5340a2d30 msgr2=0x7fc5340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc5340a2d30 0x7fc5340a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538005fb0 comp rx=0 tx=0).stop 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 msgr2=0x7fc54c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 --2- 172.21.15.111:0/1170318393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc54c1045b0 0x7fc54c1a0790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc54000eb30 tx=0x7fc54000ee40 comp rx=0 tx=0).stop 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552400700 1 -- 172.21.15.111:0/1170318393 reap_dead start 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 shutdown_connections 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 >> 172.21.15.111:0/1170318393 conn(0x7fc54c0ffe20 msgr2=0x7fc54c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 shutdown_connections 2024-04-24T22:40:17.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:17.177+0000 7fc552e00700 1 -- 172.21.15.111:0/1170318393 wait complete. 2024-04-24T22:40:18.812 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:18 smithi111 ceph-mon[27746]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:19.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:18 smithi186 ceph-mon[32753]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:19.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:18 smithi171 ceph-mon[32337]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:19.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 -- 172.21.15.111:0/4095921011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6580103320 msgr2=0x7f6580103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 --2- 172.21.15.111:0/4095921011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6580103320 0x7f6580103700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f657000b0a0 tx=0x7f657000b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 -- 172.21.15.111:0/4095921011 shutdown_connections 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 --2- 172.21.15.111:0/4095921011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f658010c750 0x7f658010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 --2- 172.21.15.111:0/4095921011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6580103320 0x7f6580103700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 --2- 172.21.15.111:0/4095921011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 0x7f658010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 -- 172.21.15.111:0/4095921011 >> 172.21.15.111:0/4095921011 conn(0x7f65800fcb50 msgr2=0x7f65800fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 -- 172.21.15.111:0/4095921011 shutdown_connections 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.441+0000 7f6585c00700 1 -- 172.21.15.111:0/4095921011 wait complete. 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.442+0000 7f6585c00700 1 Processor -- start 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.442+0000 7f6585c00700 1 -- start start 2024-04-24T22:40:19.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6580103320 0x7f658019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 0x7f658019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 0x7f6580198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6580112510 con 0x7f658010c750 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6580112390 con 0x7f6580103320 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6580112690 con 0x7f6580103c40 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 0x7f6580198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 0x7f6580198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55366/0 (socket says 172.21.15.111:55366) 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585200700 1 -- 172.21.15.111:0/337624509 learned_addr learned my addr 172.21.15.111:0/337624509 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:19.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657fe00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6580103320 0x7f658019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 0x7f658019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 -- 172.21.15.111:0/337624509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6580103320 msgr2=0x7f658019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6580103320 0x7f658019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 -- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 msgr2=0x7f6580198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 0x7f6580198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6580198ff0 con 0x7f6580103c40 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585200700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f658010c750 0x7f6580198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657fe00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6580103320 0x7f658019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657f400700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 0x7f658019ebe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9c0 tx=0x7f656800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680095c0 con 0x7f6580103c40 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6568004d10 con 0x7f6580103c40 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.444+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65801992e0 con 0x7f6580103c40 2024-04-24T22:40:19.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.445+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568005020 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.445+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801a54c0 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.445+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6580062940 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.448+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6568004750 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657cc00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65640a2cb0 0x7f65640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f65680cf0a0 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657fe00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65640a2cb0 0x7f65640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657cc00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] conn(0x7f65640ac490 0x7f65640ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6568011e50 con 0x7f65640ac490 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.449+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f65680cf580 con 0x7f6580103c40 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.450+0000 7f6585200700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] conn(0x7f65640ac490 0x7f65640ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.450+0000 7f657fe00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65640a2cb0 0x7f65640a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6570009fd0 tx=0x7f6570006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.450+0000 7f6585200700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] conn(0x7f65640ac490 0x7f65640ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.451+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== osd.5 v2:172.21.15.186:6808/2042071677 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6568011e50 con 0x7f65640ac490 2024-04-24T22:40:19.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.469+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 --> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6580062dc0 con 0x7f65640ac490 2024-04-24T22:40:19.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f657cc00700 1 -- 172.21.15.111:0/337624509 <== osd.5 v2:172.21.15.186:6808/2042071677 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6580062dc0 con 0x7f65640ac490 2024-04-24T22:40:19.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] conn(0x7f65640ac490 msgr2=0x7f65640ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f6585c00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] conn(0x7f65640ac490 0x7f65640ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65640a2cb0 msgr2=0x7f65640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f6585c00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65640a2cb0 0x7f65640a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6570009fd0 tx=0x7f6570006210 comp rx=0 tx=0).stop 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.470+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 msgr2=0x7f658019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585c00700 1 --2- 172.21.15.111:0/337624509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6580103c40 0x7f658019ebe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9c0 tx=0x7f656800ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585200700 1 -- 172.21.15.111:0/337624509 reap_dead start 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 shutdown_connections 2024-04-24T22:40:19.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 >> 172.21.15.111:0/337624509 conn(0x7f65800fcb50 msgr2=0x7f65800fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 shutdown_connections 2024-04-24T22:40:19.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.471+0000 7f6585c00700 1 -- 172.21.15.111:0/337624509 wait complete. 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 -- 172.21.15.111:0/2511447288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 msgr2=0x7fcc44106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 --2- 172.21.15.111:0/2511447288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc44106e90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009990 tx=0x7fcc2c009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 -- 172.21.15.111:0/2511447288 shutdown_connections 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 --2- 172.21.15.111:0/2511447288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc44107570 0x7fcc4410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 --2- 172.21.15.111:0/2511447288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 --2- 172.21.15.111:0/2511447288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc44106e90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 -- 172.21.15.111:0/2511447288 >> 172.21.15.111:0/2511447288 conn(0x7fcc44079390 msgr2=0x7fcc440797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 -- 172.21.15.111:0/2511447288 shutdown_connections 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.716+0000 7fcc4b800700 1 -- 172.21.15.111:0/2511447288 wait complete. 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.717+0000 7fcc4b800700 1 Processor -- start 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.717+0000 7fcc4b800700 1 -- start start 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc4419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc4419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc44107570 0x7fcc441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112580 con 0x7fcc44102e90 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112400 con 0x7fcc44107570 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc4b800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112700 con 0x7fcc441024e0 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc3be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc4419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc48e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc4419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc48e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc4419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:47496/0 (socket says 172.21.15.111:47496) 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc48e00700 1 -- 172.21.15.111:0/4157195806 learned_addr learned my addr 172.21.15.111:0/4157195806 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc3be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc4419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55330/0 (socket says 172.21.15.111:55330) 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc49800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc44107570 0x7fcc441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.718+0000 7fcc48e00700 1 -- 172.21.15.111:0/4157195806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc44107570 msgr2=0x7fcc441988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc48e00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc44107570 0x7fcc441988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc48e00700 1 -- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 msgr2=0x7fcc4419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc48e00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc4419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc48e00700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc44199000 con 0x7fcc441024e0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc3be00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc44102e90 0x7fcc4419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc48e00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc4419e6e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9c0 tx=0x7fcc3400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400cbd0 con 0x7fcc441024e0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc3400ee50 con 0x7fcc441024e0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34005680 con 0x7fcc441024e0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc441992f0 con 0x7fcc441024e0 2024-04-24T22:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.719+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc441a55b0 con 0x7fcc441024e0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.720+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc44062980 con 0x7fcc441024e0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.722+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcc34004750 con 0x7fcc441024e0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc42000700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcc280a2d30 0x7fcc280a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcc340cf760 con 0x7fcc441024e0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc3be00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcc280a2d30 0x7fcc280a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc42000700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] conn(0x7fcc280ac510 0x7fcc280ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc3400cdc0 con 0x7fcc280ac510 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcc340cfb10 con 0x7fcc441024e0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.724+0000 7fcc49800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] conn(0x7fcc280ac510 0x7fcc280ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.725+0000 7fcc3be00700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcc280a2d30 0x7fcc280a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc44107080 tx=0x7fcc2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.725+0000 7fcc49800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] conn(0x7fcc280ac510 0x7fcc280ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.726+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== osd.9 v2:172.21.15.111:6826/637881617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc3400cdc0 con 0x7fcc280ac510 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.744+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 --> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc44062e00 con 0x7fcc280ac510 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc42000700 1 -- 172.21.15.111:0/4157195806 <== osd.9 v2:172.21.15.111:6826/637881617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc44062e00 con 0x7fcc280ac510 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] conn(0x7fcc280ac510 msgr2=0x7fcc280ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] conn(0x7fcc280ac510 0x7fcc280ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcc280a2d30 msgr2=0x7fcc280a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fcc280a2d30 0x7fcc280a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc44107080 tx=0x7fcc2c005fd0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 msgr2=0x7fcc4419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 --2- 172.21.15.111:0/4157195806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc441024e0 0x7fcc4419e6e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9c0 tx=0x7fcc3400ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc49800700 1 -- 172.21.15.111:0/4157195806 reap_dead start 2024-04-24T22:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 shutdown_connections 2024-04-24T22:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.745+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 >> 172.21.15.111:0/4157195806 conn(0x7fcc44079390 msgr2=0x7fcc441086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.746+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 shutdown_connections 2024-04-24T22:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:18.746+0000 7fcc4b800700 1 -- 172.21.15.111:0/4157195806 wait complete. 2024-04-24T22:40:20.482 INFO:teuthology.orchestra.run.smithi111.stdout:219043332142 2024-04-24T22:40:20.482 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T22:40:20.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:20 smithi111 ceph-mon[27746]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T22:40:20.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 -- 172.21.15.111:0/883472543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074d80 msgr2=0x7fa30c10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 --2- 172.21.15.111:0/883472543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074d80 0x7fa30c10e6f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80098f0 tx=0x7fa2f8009c00 comp rx=0 tx=0).stop 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 -- 172.21.15.111:0/883472543 shutdown_connections 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 --2- 172.21.15.111:0/883472543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c10ec30 0x7fa30c111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 --2- 172.21.15.111:0/883472543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa30c074460 0x7fa30c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 --2- 172.21.15.111:0/883472543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074d80 0x7fa30c10e6f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.229+0000 7fa312800700 1 -- 172.21.15.111:0/883472543 >> 172.21.15.111:0/883472543 conn(0x7fa30c079390 msgr2=0x7fa30c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.230+0000 7fa312800700 1 -- 172.21.15.111:0/883472543 shutdown_connections 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.230+0000 7fa312800700 1 -- 172.21.15.111:0/883472543 wait complete. 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 Processor -- start 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 -- start start 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 0x7fa30c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa30c074d80 0x7fa30c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c10ec30 0x7fa30c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c114790 con 0x7fa30c10ec30 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c114610 con 0x7fa30c074460 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.231+0000 7fa312800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c114910 con 0x7fa30c074d80 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 0x7fa30c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c10ec30 0x7fa30c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 0x7fa30c1a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44612/0 (socket says 172.21.15.111:44612) 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 -- 172.21.15.111:0/4134992299 learned_addr learned my addr 172.21.15.111:0/4134992299 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:20.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30aa00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa30c074d80 0x7fa30c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 -- 172.21.15.111:0/4134992299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa30c074d80 msgr2=0x7fa30c1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa30c074d80 0x7fa30c1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 -- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c10ec30 msgr2=0x7fa30c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c10ec30 0x7fa30c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c19b1f0 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.232+0000 7fa30b400700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 0x7fa30c1a08d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00e9a0 tx=0x7fa2fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.233+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00ca10 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.233+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc00cbb0 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.233+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c19b4e0 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.233+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc018680 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.233+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1a77a0 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.234+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa30c0629c0 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.237+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa2fc010430 con 0x7fa30c074460 2024-04-24T22:40:20.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.238+0000 7fa303e00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa2f00a2d30 0x7fa2f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa2fc0cfaf0 con 0x7fa30c074460 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa30aa00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa2f00a2d30 0x7fa2f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa303e00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] conn(0x7fa2f00ac510 0x7fa2f00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2fc01bec0 con 0x7fa2f00ac510 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc0cfea0 con 0x7fa30c074460 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa30be00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] conn(0x7fa2f00ac510 0x7fa2f00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.239+0000 7fa30aa00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa2f00a2d30 0x7fa2f00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80098c0 tx=0x7fa2f8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.240+0000 7fa30be00700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] conn(0x7fa2f00ac510 0x7fa2f00ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.241+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== osd.0 v2:172.21.15.111:6802/1011608049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2fc01bec0 con 0x7fa2f00ac510 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.261+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 --> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa30c062dc0 con 0x7fa2f00ac510 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.261+0000 7fa303e00700 1 -- 172.21.15.111:0/4134992299 <== osd.0 v2:172.21.15.111:6802/1011608049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa30c062dc0 con 0x7fa2f00ac510 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] conn(0x7fa2f00ac510 msgr2=0x7fa2f00ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] conn(0x7fa2f00ac510 0x7fa2f00ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa2f00a2d30 msgr2=0x7fa2f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa2f00a2d30 0x7fa2f00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80098c0 tx=0x7fa2f8019040 comp rx=0 tx=0).stop 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 msgr2=0x7fa30c1a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 --2- 172.21.15.111:0/4134992299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa30c074460 0x7fa30c1a08d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00e9a0 tx=0x7fa2fc00ecb0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa30be00700 1 -- 172.21.15.111:0/4134992299 reap_dead start 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 shutdown_connections 2024-04-24T22:40:20.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 >> 172.21.15.111:0/4134992299 conn(0x7fa30c079390 msgr2=0x7fa30c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 shutdown_connections 2024-04-24T22:40:20.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.262+0000 7fa312800700 1 -- 172.21.15.111:0/4134992299 wait complete. 2024-04-24T22:40:20.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:20 smithi186 ceph-mon[32753]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T22:40:21.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:20 smithi171 ceph-mon[32337]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T22:40:21.693 INFO:teuthology.orchestra.run.smithi111.stdout:287762808866 2024-04-24T22:40:21.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T22:40:21.758 INFO:teuthology.orchestra.run.smithi111.stdout:107374182467 2024-04-24T22:40:21.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T22:40:21.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 -- 172.21.15.111:0/3704340645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0138102e90 msgr2=0x7f0138106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 --2- 172.21.15.111:0/3704340645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0138102e90 0x7f0138106e90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0120009990 tx=0x7f0120009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 -- 172.21.15.111:0/3704340645 shutdown_connections 2024-04-24T22:40:21.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 --2- 172.21.15.111:0/3704340645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 0x7f013810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 --2- 172.21.15.111:0/3704340645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01381024e0 0x7f01381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 --2- 172.21.15.111:0/3704340645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0138102e90 0x7f0138106e90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 -- 172.21.15.111:0/3704340645 >> 172.21.15.111:0/3704340645 conn(0x7f0138079390 msgr2=0x7f01380797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 -- 172.21.15.111:0/3704340645 shutdown_connections 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.125+0000 7f013e400700 1 -- 172.21.15.111:0/3704340645 wait complete. 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.126+0000 7f013e400700 1 Processor -- start 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.126+0000 7f013e400700 1 -- start start 2024-04-24T22:40:21.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 0x7f01381a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 0x7f01381a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 0x7f013819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112580 con 0x7f0138102e90 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112400 con 0x7f01381024e0 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f013e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112700 con 0x7f0138107570 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f012fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 0x7f01381a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f0137400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 0x7f01381a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f012fe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 0x7f01381a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55340/0 (socket says 172.21.15.111:55340) 2024-04-24T22:40:21.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f0137400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 0x7f01381a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44596/0 (socket says 172.21.15.111:44596) 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f012fe00700 1 -- 172.21.15.111:0/3356433015 learned_addr learned my addr 172.21.15.111:0/3356433015 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.127+0000 7f0137e00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 0x7f013819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 -- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 msgr2=0x7f013819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 0x7f013819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 -- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 msgr2=0x7f01381a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 0x7f01381a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013819b1e0 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f012fe00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0138102e90 0x7f01381a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137e00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138107570 0x7f013819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0137400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 0x7f01381a08c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f012800e9f0 tx=0x7f012800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012800cbd0 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012800cd70 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128018710 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013819b4d0 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.128+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a7790 con 0x7f01381024e0 2024-04-24T22:40:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.129+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01380757c0 con 0x7f01381024e0 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.131+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0128010430 con 0x7f01381024e0 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f0134c00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f011c0a2cb0 0x7f011c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f01280cfaf0 con 0x7f01381024e0 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f012fe00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f011c0a2cb0 0x7f011c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f0134c00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] conn(0x7f011c0ac490 0x7f011c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f011c0aef60 con 0x7f011c0ac490 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.133+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f01280cfea0 con 0x7f01381024e0 2024-04-24T22:40:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.134+0000 7f0137e00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] conn(0x7f011c0ac490 0x7f011c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.134+0000 7f012fe00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f011c0a2cb0 0x7f011c0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0120009750 tx=0x7f012000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.134+0000 7f0137e00700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] conn(0x7f011c0ac490 0x7f011c0ae8b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.135+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== osd.11 v2:172.21.15.186:6824/33009577 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f011c0aef60 con 0x7f011c0ac490 2024-04-24T22:40:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.153+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 --> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0138103970 con 0x7f011c0ac490 2024-04-24T22:40:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f0134c00700 1 -- 172.21.15.111:0/3356433015 <== osd.11 v2:172.21.15.186:6824/33009577 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0138103970 con 0x7f011c0ac490 2024-04-24T22:40:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] conn(0x7f011c0ac490 msgr2=0x7f011c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] conn(0x7f011c0ac490 0x7f011c0ae8b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f011c0a2cb0 msgr2=0x7f011c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f011c0a2cb0 0x7f011c0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0120009750 tx=0x7f012000b280 comp rx=0 tx=0).stop 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 msgr2=0x7f01381a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.154+0000 7f013e400700 1 --2- 172.21.15.111:0/3356433015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01381024e0 0x7f01381a08c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f012800e9f0 tx=0x7f012800ed00 comp rx=0 tx=0).stop 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.155+0000 7f0137e00700 1 -- 172.21.15.111:0/3356433015 reap_dead start 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.155+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 shutdown_connections 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.155+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 >> 172.21.15.111:0/3356433015 conn(0x7f0138079390 msgr2=0x7f0138101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.155+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 shutdown_connections 2024-04-24T22:40:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:19.155+0000 7f013e400700 1 -- 172.21.15.111:0/3356433015 wait complete. 2024-04-24T22:40:22.357 INFO:teuthology.orchestra.run.smithi111.stdout:34359738454 2024-04-24T22:40:22.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T22:40:22.892 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:22 smithi111 ceph-mon[27746]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T22:40:22.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:22 smithi186 ceph-mon[32753]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T22:40:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:22 smithi171 ceph-mon[32337]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T22:40:23.471 INFO:teuthology.orchestra.run.smithi111.stdout:343597383703 2024-04-24T22:40:23.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T22:40:23.847 INFO:teuthology.orchestra.run.smithi111.stdout:55834574928 2024-04-24T22:40:23.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T22:40:24.739 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:24.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 -- 172.21.15.111:0/515398416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8107570 msgr2=0x7f86e810f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 --2- 172.21.15.111:0/515398416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8107570 0x7f86e810f140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b0a0 tx=0x7f86e000b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 -- 172.21.15.111:0/515398416 shutdown_connections 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 --2- 172.21.15.111:0/515398416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8107570 0x7f86e810f140 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 --2- 172.21.15.111:0/515398416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 --2- 172.21.15.111:0/515398416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86e8102e90 0x7f86e8106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 -- 172.21.15.111:0/515398416 >> 172.21.15.111:0/515398416 conn(0x7f86e8079390 msgr2=0x7f86e80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 -- 172.21.15.111:0/515398416 shutdown_connections 2024-04-24T22:40:24.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.792+0000 7f86ef800700 1 -- 172.21.15.111:0/515398416 wait complete. 2024-04-24T22:40:24.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.793+0000 7f86ef800700 1 Processor -- start 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.793+0000 7f86ef800700 1 -- start start 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e8074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8102e90 0x7f86e8074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86e8107570 0x7f86e8074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8112570 con 0x7f86e8102e90 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81123f0 con 0x7f86e81024e0 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ef800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81126f0 con 0x7f86e8107570 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ece00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e8074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ece00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e8074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44650/0 (socket says 172.21.15.111:44650) 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ece00700 1 -- 172.21.15.111:0/816133712 learned_addr learned my addr 172.21.15.111:0/816133712 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86e7e00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8102e90 0x7f86e8074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.794+0000 7f86ed800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86e8107570 0x7f86e8074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 -- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86e8107570 msgr2=0x7f86e8074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86e8107570 0x7f86e8074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 -- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8102e90 msgr2=0x7f86e8074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8102e90 0x7f86e8074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81a6f90 con 0x7f86e81024e0 2024-04-24T22:40:24.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86e7e00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e8102e90 0x7f86e8074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:24.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.795+0000 7f86ece00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e8074530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e9c0 tx=0x7f86d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.796+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400cbd0 con 0x7f86e81024e0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.796+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d400cd70 con 0x7f86e81024e0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.796+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81a7280 con 0x7f86e81024e0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.796+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4010430 con 0x7f86e81024e0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.796+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a77c0 con 0x7f86e81024e0 2024-04-24T22:40:24.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.797+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86e8062940 con 0x7f86e81024e0 2024-04-24T22:40:24.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.799+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f86d4044b60 con 0x7f86e81024e0 2024-04-24T22:40:24.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.800+0000 7f86e5600700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86cc0a2db0 0x7f86cc0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.800+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f86d40cf530 con 0x7f86e81024e0 2024-04-24T22:40:24.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86e5600700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] conn(0x7f86cc0ac590 0x7f86cc0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86e7e00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86cc0a2db0 0x7f86cc0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86d4004500 con 0x7f86cc0ac590 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f86d40cf920 con 0x7f86e81024e0 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86ed800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] conn(0x7f86cc0ac590 0x7f86cc0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.801+0000 7f86e7e00700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86cc0a2db0 0x7f86cc0a5270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86e8108aa0 tx=0x7f86d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.802+0000 7f86ed800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] conn(0x7f86cc0ac590 0x7f86cc0ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.802+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== osd.8 v2:172.21.15.186:6816/796165885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86d4004500 con 0x7f86cc0ac590 2024-04-24T22:40:24.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.820+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 --> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86e8062d40 con 0x7f86cc0ac590 2024-04-24T22:40:24.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.821+0000 7f86e5600700 1 -- 172.21.15.111:0/816133712 <== osd.8 v2:172.21.15.186:6816/796165885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86e8062d40 con 0x7f86cc0ac590 2024-04-24T22:40:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.821+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] conn(0x7f86cc0ac590 msgr2=0x7f86cc0ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.821+0000 7f86ef800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] conn(0x7f86cc0ac590 0x7f86cc0ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86cc0a2db0 msgr2=0x7f86cc0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86cc0a2db0 0x7f86cc0a5270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86e8108aa0 tx=0x7f86d8009260 comp rx=0 tx=0).stop 2024-04-24T22:40:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 msgr2=0x7f86e8074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 --2- 172.21.15.111:0/816133712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86e81024e0 0x7f86e8074530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e9c0 tx=0x7f86d400ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ed800700 1 -- 172.21.15.111:0/816133712 reap_dead start 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 shutdown_connections 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 >> 172.21.15.111:0/816133712 conn(0x7f86e8079390 msgr2=0x7f86e8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 shutdown_connections 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:24.822+0000 7f86ef800700 1 -- 172.21.15.111:0/816133712 wait complete. 2024-04-24T22:40:24.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:24 smithi111 ceph-mon[27746]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T22:40:24.892 INFO:teuthology.orchestra.run.smithi111.stdout:163208757306 2024-04-24T22:40:24.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T22:40:24.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:24 smithi186 ceph-mon[32753]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T22:40:25.109 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:25.110 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:25.111 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:25.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:24 smithi171 ceph-mon[32337]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T22:40:25.878 INFO:teuthology.orchestra.run.smithi111.stdout:253403070505 2024-04-24T22:40:25.878 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T22:40:26.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:26 smithi186 ceph-mon[32753]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:27.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:26 smithi111 ceph-mon[27746]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:27.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:26 smithi171 ceph-mon[32337]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:27.355 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:27.546 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:27.559 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:27.560 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:27.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-24T22:40:27.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T22:40:27.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T22:40:27.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T22:40:27.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-24T22:40:27.831 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T22:40:27.831 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T22:40:27.831 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:27 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-24T22:40:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T22:40:28.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:27 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T22:40:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:27 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-24T22:40:28.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:28.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-24T22:40:28.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T22:40:28.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-24T22:40:28.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:28 smithi111 ceph-mon[27746]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T22:40:28.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-24T22:40:29.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:28 smithi186 ceph-mon[32753]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-24T22:40:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T22:40:29.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T22:40:29.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-24T22:40:29.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:28 smithi171 ceph-mon[32337]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T22:40:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:29 smithi186 ceph-mon[32753]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T22:40:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:29 smithi111 ceph-mon[27746]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T22:40:30.058 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:29 smithi171 ceph-mon[32337]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T22:40:30.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:30 smithi186 ceph-mon[32753]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T22:40:31.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:30 smithi111 ceph-mon[27746]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T22:40:31.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:30 smithi171 ceph-mon[32337]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T22:40:32.916 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:32 smithi171 ceph-mon[32337]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-04-24T22:40:32.916 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:32 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:32 smithi186 ceph-mon[32753]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-04-24T22:40:32.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:32 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:33.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:32 smithi111 ceph-mon[27746]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-04-24T22:40:33.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:32 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:34.837 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.837 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.837 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:34 smithi111 ceph-mon[27746]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:34.837 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.837 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:34 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:34 smithi171 ceph-mon[32337]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:34.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:34 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:34 smithi186 ceph-mon[32753]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:34.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:35.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:34 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:35.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 -- 172.21.15.111:0/1922604259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 msgr2=0x7f750c111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 --2- 172.21.15.111:0/1922604259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c111320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f750400b0a0 tx=0x7f750400b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 -- 172.21.15.111:0/1922604259 shutdown_connections 2024-04-24T22:40:35.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 --2- 172.21.15.111:0/1922604259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c111320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 --2- 172.21.15.111:0/1922604259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 0x7f750c10a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 --2- 172.21.15.111:0/1922604259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 -- 172.21.15.111:0/1922604259 >> 172.21.15.111:0/1922604259 conn(0x7f750c0fff40 msgr2=0x7f750c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 -- 172.21.15.111:0/1922604259 shutdown_connections 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.004+0000 7f7510e00700 1 -- 172.21.15.111:0/1922604259 wait complete. 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.005+0000 7f7510e00700 1 Processor -- start 2024-04-24T22:40:35.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.005+0000 7f7510e00700 1 -- start start 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 0x7f750c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f750c114790 con 0x7f750c10a4e0 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f750c114610 con 0x7f750c104450 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7510e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f750c114910 con 0x7f750c104e10 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7509e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7509400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7509e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:54318/0 (socket says 172.21.15.111:54318) 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f7509e00700 1 -- 172.21.15.111:0/2809815162 learned_addr learned my addr 172.21.15.111:0/2809815162 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:35.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.006+0000 7f750a800700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 0x7f750c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 -- 172.21.15.111:0/2809815162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 msgr2=0x7f750c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 -- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 msgr2=0x7f750c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 0x7f750c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 -- 172.21.15.111:0/2809815162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c19b1f0 con 0x7f750c104e10 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7509400700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c1a0d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d750 tx=0x7f74f800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8014030 con 0x7f750c104e10 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f8004500 con 0x7f750c104e10 2024-04-24T22:40:35.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.007+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8005420 con 0x7f750c104e10 2024-04-24T22:40:35.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.008+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c19b4e0 con 0x7f750c104e10 2024-04-24T22:40:35.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.008+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c1a76b0 con 0x7f750c104e10 2024-04-24T22:40:35.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.009+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c068f20 con 0x7f750c104e10 2024-04-24T22:40:35.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.011+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f74f8004750 con 0x7f750c104e10 2024-04-24T22:40:35.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.013+0000 7f7502a00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 0x7f74f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.013+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f74f80cede0 con 0x7f750c104e10 2024-04-24T22:40:35.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.013+0000 7f7509e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 0x7f74f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.014+0000 7f7509e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 0x7f74f00a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f750c102700 tx=0x7f74fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:35.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.017+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f74f809bb90 con 0x7f750c104e10 2024-04-24T22:40:35.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.254+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f750c02dcf0 con 0x7f750c104e10 2024-04-24T22:40:35.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.254+0000 7f7502a00700 1 -- 172.21.15.111:0/2809815162 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74f809b490 con 0x7f750c104e10 2024-04-24T22:40:35.256 INFO:teuthology.orchestra.run.smithi111.stdout:197568495670 2024-04-24T22:40:35.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 msgr2=0x7f74f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 0x7f74f00a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f750c102700 tx=0x7f74fc009260 comp rx=0 tx=0).stop 2024-04-24T22:40:35.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 msgr2=0x7f750c1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c1a0d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d750 tx=0x7f74f800da60 comp rx=0 tx=0).stop 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 shutdown_connections 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f750c10a4e0 0x7f750c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f74f00a2d30 0x7f74f00a51f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f750c104450 0x7f750c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 --2- 172.21.15.111:0/2809815162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f750c104e10 0x7f750c1a0d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 >> 172.21.15.111:0/2809815162 conn(0x7f750c0fff40 msgr2=0x7f750c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 shutdown_connections 2024-04-24T22:40:35.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.257+0000 7f7510e00700 1 -- 172.21.15.111:0/2809815162 wait complete. 2024-04-24T22:40:35.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 -- 172.21.15.111:0/4183376752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006a7d0 msgr2=0x7fb1d006ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/4183376752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006a7d0 0x7fb1d006ac30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009990 tx=0x7fb1bc009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 -- 172.21.15.111:0/4183376752 shutdown_connections 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/4183376752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006b170 0x7fb1d010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/4183376752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 0x7fb1d0108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/4183376752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006a7d0 0x7fb1d006ac30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 -- 172.21.15.111:0/4183376752 >> 172.21.15.111:0/4183376752 conn(0x7fb1d00780a0 msgr2=0x7fb1d00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 -- 172.21.15.111:0/4183376752 shutdown_connections 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.405+0000 7fb1d7e00700 1 -- 172.21.15.111:0/4183376752 wait complete. 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.406+0000 7fb1d7e00700 1 Processor -- start 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 -- start start 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006a7d0 0x7fb1d01011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 0x7fb1d0101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0112550 con 0x7fb1d006a7d0 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01123d0 con 0x7fb1d01082a0 2024-04-24T22:40:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.407+0000 7fb1d7e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01126d0 con 0x7fb1d006b170 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d5400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006a7d0 0x7fb1d01011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56776/0 (socket says 172.21.15.111:56776) 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 -- 172.21.15.111:0/3010958007 learned_addr learned my addr 172.21.15.111:0/3010958007 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d5e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 0x7fb1d0101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 -- 172.21.15.111:0/3010958007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 msgr2=0x7fb1d0101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 0x7fb1d0101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 -- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006a7d0 msgr2=0x7fb1d01011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006a7d0 0x7fb1d01011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 -- 172.21.15.111:0/3010958007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d01a6f50 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.408+0000 7fb1d4a00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc000c00 tx=0x7fb1bc00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.409+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc027070 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.409+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc003d70 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.409+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00f460 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.409+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d01a71e0 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.409+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01a75e0 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.410+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d0068f20 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.412+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb1bc02a070 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.413+0000 7fb1ce000700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 0x7fb1b80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.414+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb1bc0cfff0 con 0x7fb1d006b170 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.414+0000 7fb1d5400700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 0x7fb1b80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.414+0000 7fb1d5400700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 0x7fb1b80a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009be0 tx=0x7fb1c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:35.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.417+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1bc05e740 con 0x7fb1d006b170 2024-04-24T22:40:35.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.649+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb1d002dd90 con 0x7fb1d006b170 2024-04-24T22:40:35.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.650+0000 7fb1ce000700 1 -- 172.21.15.111:0/3010958007 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1bc013070 con 0x7fb1d006b170 2024-04-24T22:40:35.653 INFO:teuthology.orchestra.run.smithi111.stdout:141733920834 2024-04-24T22:40:35.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 msgr2=0x7fb1b80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 0x7fb1b80a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009be0 tx=0x7fb1c4009260 comp rx=0 tx=0).stop 2024-04-24T22:40:35.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 msgr2=0x7fb1d0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:35.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc000c00 tx=0x7fb1bc00bab0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 shutdown_connections 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb1b80a2d30 0x7fb1b80a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb1d01082a0 0x7fb1d0101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d006a7d0 0x7fb1d01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 --2- 172.21.15.111:0/3010958007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d006b170 0x7fb1d0101730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 >> 172.21.15.111:0/3010958007 conn(0x7fb1d00780a0 msgr2=0x7fb1d00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 shutdown_connections 2024-04-24T22:40:35.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.653+0000 7fb1d7e00700 1 -- 172.21.15.111:0/3010958007 wait complete. 2024-04-24T22:40:35.673 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:35 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2809815162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T22:40:35.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:35 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2809815162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T22:40:35.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:35 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2809815162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T22:40:36.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:36 smithi171 ceph-mon[32337]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:36.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:36 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3010958007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T22:40:36.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:36 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1131361898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T22:40:36.975 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:36 smithi111 ceph-mon[27746]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:36.975 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:36 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3010958007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T22:40:36.975 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:36 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1131361898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T22:40:36.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:36 smithi186 ceph-mon[32753]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:37.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:36 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3010958007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T22:40:37.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:36 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1131361898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T22:40:37.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 -- 172.21.15.111:0/2317934618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 msgr2=0x7efedc06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:37.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 --2- 172.21.15.111:0/2317934618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc06a3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 -- 172.21.15.111:0/2317934618 shutdown_connections 2024-04-24T22:40:37.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 --2- 172.21.15.111:0/2317934618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 0x7efedc10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 --2- 172.21.15.111:0/2317934618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 0x7efedc108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 --2- 172.21.15.111:0/2317934618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc06a3a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 -- 172.21.15.111:0/2317934618 >> 172.21.15.111:0/2317934618 conn(0x7efedc0780e0 msgr2=0x7efedc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 -- 172.21.15.111:0/2317934618 shutdown_connections 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.509+0000 7efee3800700 1 -- 172.21.15.111:0/2317934618 wait complete. 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.510+0000 7efee3800700 1 Processor -- start 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.510+0000 7efee3800700 1 -- start start 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 0x7efedc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 0x7efedc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efedc112560 con 0x7efedc06a8e0 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1123e0 con 0x7efedc1082b0 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee3800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1126e0 con 0x7efedc069f40 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:37.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56796/0 (socket says 172.21.15.111:56796) 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 -- 172.21.15.111:0/1131361898 learned_addr learned my addr 172.21.15.111:0/1131361898 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee1800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 0x7efedc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efedbe00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 0x7efedc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 -- 172.21.15.111:0/1131361898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 msgr2=0x7efedc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 0x7efedc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 -- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 msgr2=0x7efedc1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 0x7efedc1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.511+0000 7efee0e00700 1 -- 172.21.15.111:0/1131361898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc19b1a0 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efee0e00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed000d950 tx=0x7efed000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0016070 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed000fd70 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc19b490 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1a7750 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.512+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000e610 con 0x7efedc069f40 2024-04-24T22:40:37.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.513+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc005330 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.515+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efed0019070 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.516+0000 7efed9600700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 0x7efec40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.516+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efed00cee80 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.517+0000 7efedbe00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 0x7efec40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.517+0000 7efedbe00700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 0x7efec40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efedc10f510 tx=0x7efec8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.520+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efed009bc30 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.749+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7efebc0058f0 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.750+0000 7efed9600700 1 -- 172.21.15.111:0/1131361898 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7efed009b530 con 0x7efedc069f40 2024-04-24T22:40:37.195 INFO:teuthology.orchestra.run.smithi111.stdout:309237645343 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.752+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 msgr2=0x7efec40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.752+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 0x7efec40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efedc10f510 tx=0x7efec8005fd0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.752+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 msgr2=0x7efedc1a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.752+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed000d950 tx=0x7efed000dc60 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 shutdown_connections 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7efec40a2d30 0x7efec40a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efedc1082b0 0x7efedc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efedc069f40 0x7efedc1a0880 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 --2- 172.21.15.111:0/1131361898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc06a8e0 0x7efedc1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 >> 172.21.15.111:0/1131361898 conn(0x7efedc0780e0 msgr2=0x7efedc0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 shutdown_connections 2024-04-24T22:40:37.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:35.753+0000 7efee3800700 1 -- 172.21.15.111:0/1131361898 wait complete. 2024-04-24T22:40:38.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920834 for osd.4 2024-04-24T22:40:38.559 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:38.726 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:38 smithi111 ceph-mon[27746]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:38.914 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:38 smithi171 ceph-mon[32337]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:39.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:38 smithi186 ceph-mon[32753]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T22:40:39.655 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 -- 172.21.15.111:0/1599028771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 msgr2=0x7f75a41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 --2- 172.21.15.111:0/1599028771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a41027a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f75940099c0 tx=0x7f7594009cd0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 -- 172.21.15.111:0/1599028771 shutdown_connections 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 --2- 172.21.15.111:0/1599028771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a410f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 --2- 172.21.15.111:0/1599028771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a41027a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 --2- 172.21.15.111:0/1599028771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.021+0000 7f75a8a00700 1 -- 172.21.15.111:0/1599028771 >> 172.21.15.111:0/1599028771 conn(0x7f75a40fdcb0 msgr2=0x7f75a41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:40.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.022+0000 7f75a8a00700 1 -- 172.21.15.111:0/1599028771 shutdown_connections 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.022+0000 7f75a8a00700 1 -- 172.21.15.111:0/1599028771 wait complete. 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 Processor -- start 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 -- start start 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41124d0 con 0x7f75a41023c0 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112350 con 0x7f75a4102d70 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a8a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112650 con 0x7f75a41074a0 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40806/0 (socket says 172.21.15.111:40806) 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.023+0000 7f75a3400700 1 -- 172.21.15.111:0/2050798961 learned_addr learned my addr 172.21.15.111:0/2050798961 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a2a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3e00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 -- 172.21.15.111:0/2050798961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 msgr2=0x7f75a4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 -- 172.21.15.111:0/2050798961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 msgr2=0x7f75a419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 -- 172.21.15.111:0/2050798961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a4199030 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a2a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3e00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a3400700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7594006010 tx=0x7f7594003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75940050c0 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759400f460 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.024+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a41992c0 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.025+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594017650 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.025+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a41a5430 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.026+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a41038a0 con 0x7f75a41023c0 2024-04-24T22:40:40.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.028+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f759400f610 con 0x7f75a41023c0 2024-04-24T22:40:40.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.029+0000 7f759be00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 0x7f75880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.029+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f75940d01b0 con 0x7f75a41023c0 2024-04-24T22:40:40.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.029+0000 7f75a2a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 0x7f75880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.030+0000 7f75a2a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 0x7f75880a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f75a4199f10 tx=0x7f7590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:40.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.034+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7594013070 con 0x7f75a41023c0 2024-04-24T22:40:40.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.261+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f75a402db30 con 0x7f75a41023c0 2024-04-24T22:40:40.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.261+0000 7f759be00700 1 -- 172.21.15.111:0/2050798961 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f759409cff0 con 0x7f75a41023c0 2024-04-24T22:40:40.263 INFO:teuthology.orchestra.run.smithi111.stdout:343597383707 2024-04-24T22:40:40.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 msgr2=0x7f75880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 0x7f75880a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f75a4199f10 tx=0x7f7590009260 comp rx=0 tx=0).stop 2024-04-24T22:40:40.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 msgr2=0x7f75a419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7594006010 tx=0x7f7594003b40 comp rx=0 tx=0).stop 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 shutdown_connections 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75a41074a0 0x7f75a4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75a41023c0 0x7f75a419e680 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f75880a2cb0 0x7f75880a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 --2- 172.21.15.111:0/2050798961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75a4102d70 0x7f75a419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.265+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 >> 172.21.15.111:0/2050798961 conn(0x7f75a40fdcb0 msgr2=0x7f75a40fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.266+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 shutdown_connections 2024-04-24T22:40:40.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.266+0000 7f75a8a00700 1 -- 172.21.15.111:0/2050798961 wait complete. 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 -- 172.21.15.111:0/2202375839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 msgr2=0x7f0e9810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 --2- 172.21.15.111:0/2202375839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e9810e750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098f0 tx=0x7f0e88009c00 comp rx=0 tx=0).stop 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 -- 172.21.15.111:0/2202375839 shutdown_connections 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 --2- 172.21.15.111:0/2202375839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e9810ec90 0x7f0e98111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 --2- 172.21.15.111:0/2202375839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 0x7f0e98074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 --2- 172.21.15.111:0/2202375839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e9810e750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 -- 172.21.15.111:0/2202375839 >> 172.21.15.111:0/2202375839 conn(0x7f0e98079390 msgr2=0x7f0e980797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 -- 172.21.15.111:0/2202375839 shutdown_connections 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.461+0000 7f0ea0800700 1 -- 172.21.15.111:0/2202375839 wait complete. 2024-04-24T22:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 Processor -- start 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 -- start start 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 0x7f0e981a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e9810ec90 0x7f0e9819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e981147d0 con 0x7f0e98074d80 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98114650 con 0x7f0e98074460 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.463+0000 7f0ea0800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98114950 con 0x7f0e9810ec90 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40816/0 (socket says 172.21.15.111:40816) 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 -- 172.21.15.111:0/1956230867 learned_addr learned my addr 172.21.15.111:0/1956230867 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9de00700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 0x7f0e981a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 -- 172.21.15.111:0/1956230867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e9810ec90 msgr2=0x7f0e9819aa50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e9810ec90 0x7f0e9819aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 -- 172.21.15.111:0/1956230867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 msgr2=0x7f0e981a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 0x7f0e981a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 -- 172.21.15.111:0/1956230867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9819b1b0 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.464+0000 7f0e9d400700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e8800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.465+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e880051f0 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.465+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8800b560 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.465+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88003ae0 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.465+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9819b440 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.465+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e981a7760 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.466+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e98075b20 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.468+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0e8800ba80 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.470+0000 7f0e92a00700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 0x7f0e800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.470+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0e880d0160 con 0x7f0e98074d80 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.470+0000 7f0e9de00700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 0x7f0e800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.471+0000 7f0e9de00700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 0x7f0e800a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e9819bb90 tx=0x7f0e8c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:40.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.475+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e88013030 con 0x7f0e98074d80 2024-04-24T22:40:40.484 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:40.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.709+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f0e981a7930 con 0x7f0e98074d80 2024-04-24T22:40:40.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.710+0000 7f0e92a00700 1 -- 172.21.15.111:0/1956230867 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e88021380 con 0x7f0e98074d80 2024-04-24T22:40:40.712 INFO:teuthology.orchestra.run.smithi111.stdout:287762808870 2024-04-24T22:40:40.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 msgr2=0x7f0e800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 0x7f0e800a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e9819bb90 tx=0x7f0e8c009310 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 msgr2=0x7f0e981a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e8800fb20 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 shutdown_connections 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e9810ec90 0x7f0e9819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0e800a2d30 0x7f0e800a51f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e98074460 0x7f0e981a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 --2- 172.21.15.111:0/1956230867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e98074d80 0x7f0e981a0dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.713+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 >> 172.21.15.111:0/1956230867 conn(0x7f0e98079390 msgr2=0x7f0e980ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.714+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 shutdown_connections 2024-04-24T22:40:40.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:40.714+0000 7f0ea0800700 1 -- 172.21.15.111:0/1956230867 wait complete. 2024-04-24T22:40:40.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:40 smithi111 ceph-mon[27746]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:40.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:40 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2257070693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T22:40:40.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:40 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2050798961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T22:40:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:40 smithi171 ceph-mon[32337]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:40.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:40 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2257070693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T22:40:40.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:40 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2050798961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T22:40:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:40 smithi186 ceph-mon[32753]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:40 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2257070693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T22:40:40.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:40 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2050798961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T22:40:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:41 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1956230867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T22:40:41.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:41 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1230149834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T22:40:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:41 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1956230867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T22:40:41.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:41 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1230149834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T22:40:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 -- 172.21.15.111:0/700286302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 msgr2=0x7fdf441027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 --2- 172.21.15.111:0/700286302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441027a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf340099c0 tx=0x7fdf34009cd0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 -- 172.21.15.111:0/700286302 shutdown_connections 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 --2- 172.21.15.111:0/700286302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf441074a0 0x7fdf4410f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 --2- 172.21.15.111:0/700286302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441027a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 --2- 172.21.15.111:0/700286302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf44102d70 0x7fdf44106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 -- 172.21.15.111:0/700286302 >> 172.21.15.111:0/700286302 conn(0x7fdf440fdcb0 msgr2=0x7fdf441000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 -- 172.21.15.111:0/700286302 shutdown_connections 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.559+0000 7fdf48a00700 1 -- 172.21.15.111:0/700286302 wait complete. 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.560+0000 7fdf48a00700 1 Processor -- start 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.560+0000 7fdf48a00700 1 -- start start 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 0x7fdf441a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 0x7fdf4419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44112480 con 0x7fdf441023c0 2024-04-24T22:40:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44112300 con 0x7fdf441074a0 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf48a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44112600 con 0x7fdf44102d70 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40786/0 (socket says 172.21.15.111:40786) 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 -- 172.21.15.111:0/2257070693 learned_addr learned my addr 172.21.15.111:0/2257070693 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf41e00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 0x7fdf441a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf43200700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 0x7fdf4419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 -- 172.21.15.111:0/2257070693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 msgr2=0x7fdf441a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 0x7fdf441a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 -- 172.21.15.111:0/2257070693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 msgr2=0x7fdf4419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 0x7fdf4419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf42800700 1 -- 172.21.15.111:0/2257070693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4419b100 con 0x7fdf441023c0 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf41e00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 0x7fdf441a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.561+0000 7fdf43200700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 0x7fdf4419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf42800700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf34006010 tx=0x7fdf3400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34003680 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf340055b0 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4419b390 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340173f0 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.562+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a76b0 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.563+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf44068f20 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.565+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdf340050d0 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.567+0000 7fdf3b400700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 0x7fdf280a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.567+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdf340cfff0 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.567+0000 7fdf41e00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 0x7fdf280a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.568+0000 7fdf41e00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 0x7fdf280a5390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf440764c0 tx=0x7fdf30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.572+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf34013070 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.830+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdf441a7880 con 0x7fdf441023c0 2024-04-24T22:40:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.830+0000 7fdf3b400700 1 -- 172.21.15.111:0/2257070693 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf3409ce70 con 0x7fdf441023c0 2024-04-24T22:40:41.939 INFO:teuthology.orchestra.run.smithi111.stdout:77309411408 2024-04-24T22:40:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 msgr2=0x7fdf280a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 0x7fdf280a5390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf440764c0 tx=0x7fdf30009260 comp rx=0 tx=0).stop 2024-04-24T22:40:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 msgr2=0x7fdf441a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf34006010 tx=0x7fdf3400f740 comp rx=0 tx=0).stop 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 shutdown_connections 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdf280a2ed0 0x7fdf280a5390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf441074a0 0x7fdf4419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf441023c0 0x7fdf441a07e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 --2- 172.21.15.111:0/2257070693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf44102d70 0x7fdf441a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.833+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 >> 172.21.15.111:0/2257070693 conn(0x7fdf440fdcb0 msgr2=0x7fdf440fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.834+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 shutdown_connections 2024-04-24T22:40:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:39.834+0000 7fdf48a00700 1 -- 172.21.15.111:0/2257070693 wait complete. 2024-04-24T22:40:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:41 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1956230867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T22:40:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:41 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1230149834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T22:40:42.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 -- 172.21.15.111:0/3428480454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e010c7a0 msgr2=0x7f46e010ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 --2- 172.21.15.111:0/3428480454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e010c7a0 0x7f46e010ec60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46d800b0a0 tx=0x7f46d800b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 -- 172.21.15.111:0/3428480454 shutdown_connections 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 --2- 172.21.15.111:0/3428480454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e010c7a0 0x7f46e010ec60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 --2- 172.21.15.111:0/3428480454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103370 0x7f46e0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 --2- 172.21.15.111:0/3428480454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e0103c90 0x7f46e010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 -- 172.21.15.111:0/3428480454 >> 172.21.15.111:0/3428480454 conn(0x7f46e00fcbc0 msgr2=0x7f46e00fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 -- 172.21.15.111:0/3428480454 shutdown_connections 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.313+0000 7f46e6200700 1 -- 172.21.15.111:0/3428480454 wait complete. 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.314+0000 7f46e6200700 1 Processor -- start 2024-04-24T22:40:42.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.314+0000 7f46e6200700 1 -- start start 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103c90 0x7f46e0101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e010c7a0 0x7f46e0101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0112540 con 0x7f46e0103370 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01123c0 con 0x7f46e010c7a0 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e6200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01126c0 con 0x7f46e0103c90 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e4e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e4e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40840/0 (socket says 172.21.15.111:40840) 2024-04-24T22:40:42.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e4e00700 1 -- 172.21.15.111:0/1230149834 learned_addr learned my addr 172.21.15.111:0/1230149834 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46e5800700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e010c7a0 0x7f46e0101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.315+0000 7f46dfe00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103c90 0x7f46e0101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 -- 172.21.15.111:0/1230149834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103c90 msgr2=0x7f46e0101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103c90 0x7f46e0101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 -- 172.21.15.111:0/1230149834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e010c7a0 msgr2=0x7f46e0101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e010c7a0 0x7f46e0101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 -- 172.21.15.111:0/1230149834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e01a6ee0 con 0x7f46e0103370 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e4e00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f46d400e9a0 tx=0x7f46d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d400ca10 con 0x7f46e0103370 2024-04-24T22:40:42.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d400cbb0 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e01a71d0 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4018680 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.316+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01a7710 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.318+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e0068f20 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.319+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f46d4010430 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.321+0000 7f46dd600700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 0x7f46c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.321+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f46d40cf460 con 0x7f46e0103370 2024-04-24T22:40:42.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.321+0000 7f46dfe00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 0x7f46c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:42.384 INFO:teuthology.orchestra.run.smithi111.stdout:219043332146 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.322+0000 7f46dfe00700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 0x7f46c80a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46e010da50 tx=0x7f46cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.325+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46d409c210 con 0x7f46e0103370 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.558+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f46e002db30 con 0x7f46e0103370 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.559+0000 7f46dd600700 1 -- 172.21.15.111:0/1230149834 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f46d409bb10 con 0x7f46e0103370 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 msgr2=0x7f46c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:42.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 0x7f46c80a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46e010da50 tx=0x7f46cc009260 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 msgr2=0x7f46e0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f46d400e9a0 tx=0x7f46d400ecb0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 shutdown_connections 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46c80a2cb0 0x7f46c80a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46e010c7a0 0x7f46e0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46e0103370 0x7f46e0100fd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 --2- 172.21.15.111:0/1230149834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e0103c90 0x7f46e0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 >> 172.21.15.111:0/1230149834 conn(0x7f46e00fcbc0 msgr2=0x7f46e00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 shutdown_connections 2024-04-24T22:40:42.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:41.562+0000 7f46e6200700 1 -- 172.21.15.111:0/1230149834 wait complete. 2024-04-24T22:40:42.819 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495670 for osd.6 2024-04-24T22:40:42.819 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:42.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:42 smithi171 ceph-mon[32337]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:42.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:42 smithi186 ceph-mon[32753]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:42 smithi111 ceph-mon[27746]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T22:40:43.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411408 for osd.2 2024-04-24T22:40:43.094 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:43.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 -- 172.21.15.111:0/1693565979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 msgr2=0x7f7f24108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 --2- 172.21.15.111:0/1693565979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f24108d20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009990 tx=0x7f7f0c009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 -- 172.21.15.111:0/1693565979 shutdown_connections 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 --2- 172.21.15.111:0/1693565979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 0x7f7f24115720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 --2- 172.21.15.111:0/1693565979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 0x7f7f2410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 --2- 172.21.15.111:0/1693565979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f24108d20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 -- 172.21.15.111:0/1693565979 >> 172.21.15.111:0/1693565979 conn(0x7f7f24075a00 msgr2=0x7f7f24077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.318+0000 7f7f28a00700 1 -- 172.21.15.111:0/1693565979 shutdown_connections 2024-04-24T22:40:43.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.319+0000 7f7f28a00700 1 -- 172.21.15.111:0/1693565979 wait complete. 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 Processor -- start 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 -- start start 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 0x7f7f2419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 0x7f7f241a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24118c30 con 0x7f7f241088a0 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24118ab0 con 0x7f7f24109260 2024-04-24T22:40:43.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f28a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24118db0 con 0x7f7f2410e850 2024-04-24T22:40:43.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f21e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f21e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40858/0 (socket says 172.21.15.111:40858) 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f21e00700 1 -- 172.21.15.111:0/4148089351 learned_addr learned my addr 172.21.15.111:0/4148089351 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.320+0000 7f7f19400700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 0x7f7f2419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f22800700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 0x7f7f241a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 -- 172.21.15.111:0/4148089351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 msgr2=0x7f7f241a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 0x7f7f241a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 -- 172.21.15.111:0/4148089351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 msgr2=0x7f7f2419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 0x7f7f2419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 -- 172.21.15.111:0/4148089351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f241a4060 con 0x7f7f241088a0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f21e00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400ef30 tx=0x7f7f1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140092e0 con 0x7f7f241088a0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.321+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f14004bb0 con 0x7f7f241088a0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.322+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241a42f0 con 0x7f7f241088a0 2024-04-24T22:40:43.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.322+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14004280 con 0x7f7f241088a0 2024-04-24T22:40:43.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.322+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241abbd0 con 0x7f7f241088a0 2024-04-24T22:40:43.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.323+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24110de0 con 0x7f7f241088a0 2024-04-24T22:40:43.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.325+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7f1401b030 con 0x7f7f241088a0 2024-04-24T22:40:43.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.326+0000 7f7f1b400700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 0x7f7f080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:43.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.326+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7f140cec50 con 0x7f7f241088a0 2024-04-24T22:40:43.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.326+0000 7f7f19400700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 0x7f7f080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.327+0000 7f7f19400700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 0x7f7f080a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f240780d0 tx=0x7f7f0c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:43.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.331+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7f14022020 con 0x7f7f241088a0 2024-04-24T22:40:43.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.560+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7f241a03b0 con 0x7f7f241088a0 2024-04-24T22:40:43.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.561+0000 7f7f1b400700 1 -- 172.21.15.111:0/4148089351 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f14010040 con 0x7f7f241088a0 2024-04-24T22:40:43.563 INFO:teuthology.orchestra.run.smithi111.stdout:34359738458 2024-04-24T22:40:43.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.563+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 msgr2=0x7f7f080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:43.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.563+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 0x7f7f080a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f240780d0 tx=0x7f7f0c00b260 comp rx=0 tx=0).stop 2024-04-24T22:40:43.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.563+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 msgr2=0x7f7f2419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:43.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.563+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400ef30 tx=0x7f7f1400c3a0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 shutdown_connections 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f2410e850 0x7f7f241a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f241088a0 0x7f7f2419f0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7f080a2d30 0x7f7f080a51f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 --2- 172.21.15.111:0/4148089351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f24109260 0x7f7f2419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 >> 172.21.15.111:0/4148089351 conn(0x7f7f24075a00 msgr2=0x7f7f24076250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:43.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 shutdown_connections 2024-04-24T22:40:43.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:43.564+0000 7f7f28a00700 1 -- 172.21.15.111:0/4148089351 wait complete. 2024-04-24T22:40:43.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/4148089351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T22:40:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/4148089351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T22:40:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4148089351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T22:40:44.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808870 for osd.9 2024-04-24T22:40:44.261 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:44.730 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:44 smithi111 ceph-mon[27746]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T22:40:44.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:44 smithi171 ceph-mon[32337]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T22:40:44.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:44 smithi186 ceph-mon[32753]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 -- 172.21.15.111:0/2611818838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 msgr2=0x7f4a1c115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 --2- 172.21.15.111:0/2611818838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c115730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400b0a0 tx=0x7f4a1400b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 -- 172.21.15.111:0/2611818838 shutdown_connections 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 --2- 172.21.15.111:0/2611818838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c115730 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 --2- 172.21.15.111:0/2611818838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 0x7f4a1c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 --2- 172.21.15.111:0/2611818838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 -- 172.21.15.111:0/2611818838 >> 172.21.15.111:0/2611818838 conn(0x7f4a1c075a00 msgr2=0x7f4a1c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:45.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 -- 172.21.15.111:0/2611818838 shutdown_connections 2024-04-24T22:40:45.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.585+0000 7f4a23800700 1 -- 172.21.15.111:0/2611818838 wait complete. 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 Processor -- start 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 -- start start 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 0x7f4a1c1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c118c00 con 0x7f4a1c109220 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c118a80 con 0x7f4a1c10e860 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.586+0000 7f4a23800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c118d80 con 0x7f4a1c108860 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a1be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45760/0 (socket says 172.21.15.111:45760) 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 -- 172.21.15.111:0/2604793039 learned_addr learned my addr 172.21.15.111:0/2604793039 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a21800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 0x7f4a1c1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 -- 172.21.15.111:0/2604793039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 msgr2=0x7f4a1c1a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 0x7f4a1c1a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 -- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 msgr2=0x7f4a1c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 -- 172.21.15.111:0/2604793039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c1a3fd0 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.587+0000 7f4a20e00700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000d950 tx=0x7f4a1000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.588+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10016070 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.588+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a1000fd70 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.588+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c1a42c0 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.588+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000e610 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.588+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c1abb40 con 0x7f4a1c108860 2024-04-24T22:40:45.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.589+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c068f20 con 0x7f4a1c108860 2024-04-24T22:40:45.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.591+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4a10019070 con 0x7f4a1c108860 2024-04-24T22:40:45.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.592+0000 7f4a19600700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 0x7f4a040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.593+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4a100cec40 con 0x7f4a1c108860 2024-04-24T22:40:45.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.593+0000 7f4a1be00700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 0x7f4a040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.593+0000 7f4a1be00700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 0x7f4a040a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c077870 tx=0x7f4a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:45.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.595+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4a1009b9f0 con 0x7f4a1c108860 2024-04-24T22:40:45.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.826+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4a1c02db30 con 0x7f4a1c108860 2024-04-24T22:40:45.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.827+0000 7f4a19600700 1 -- 172.21.15.111:0/2604793039 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4a1009b2f0 con 0x7f4a1c108860 2024-04-24T22:40:45.829 INFO:teuthology.orchestra.run.smithi111.stdout:107374182472 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.829+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 msgr2=0x7f4a040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.829+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 0x7f4a040a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c077870 tx=0x7f4a08009260 comp rx=0 tx=0).stop 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.829+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 msgr2=0x7f4a1c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.829+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000d950 tx=0x7f4a1000dc60 comp rx=0 tx=0).stop 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 shutdown_connections 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4a040a2cb0 0x7f4a040a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a1c10e860 0x7f4a1c1a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a1c108860 0x7f4a1c19f010 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 --2- 172.21.15.111:0/2604793039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a1c109220 0x7f4a1c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 >> 172.21.15.111:0/2604793039 conn(0x7f4a1c075a00 msgr2=0x7f4a1c075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 shutdown_connections 2024-04-24T22:40:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:45.830+0000 7f4a23800700 1 -- 172.21.15.111:0/2604793039 wait complete. 2024-04-24T22:40:46.788 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383707 for osd.11 2024-04-24T22:40:46.788 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:46.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:46 smithi171 ceph-mon[32337]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:46.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:46 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2604793039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T22:40:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:46 smithi186 ceph-mon[32753]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:46.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:46 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2604793039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T22:40:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:46 smithi111 ceph-mon[27746]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:47.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:46 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2604793039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T22:40:47.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645343 for osd.10 2024-04-24T22:40:47.117 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:47.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332146 for osd.7 2024-04-24T22:40:47.899 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:48.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:48 smithi111 ceph-mon[27746]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:48.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738458 for osd.0 2024-04-24T22:40:48.906 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:48.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:48 smithi171 ceph-mon[32337]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:48.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:48 smithi186 ceph-mon[32753]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:49.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182472 for osd.3 2024-04-24T22:40:49.911 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:50.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:50 smithi171 ceph-mon[32337]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:40:50.960 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:50 smithi111 ceph-mon[27746]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:40:50.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:50 smithi186 ceph-mon[32753]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:40:51.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 -- 172.21.15.111:0/3017938048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c102e90 msgr2=0x7fe44c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 --2- 172.21.15.111:0/3017938048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c102e90 0x7fe44c106e90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe438009990 tx=0x7fe438009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 -- 172.21.15.111:0/3017938048 shutdown_connections 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 --2- 172.21.15.111:0/3017938048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c107570 0x7fe44c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 --2- 172.21.15.111:0/3017938048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c1024e0 0x7fe44c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 --2- 172.21.15.111:0/3017938048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c102e90 0x7fe44c106e90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 -- 172.21.15.111:0/3017938048 >> 172.21.15.111:0/3017938048 conn(0x7fe44c079390 msgr2=0x7fe44c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 -- 172.21.15.111:0/3017938048 shutdown_connections 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.526+0000 7fe452400700 1 -- 172.21.15.111:0/3017938048 wait complete. 2024-04-24T22:40:51.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.527+0000 7fe452400700 1 Processor -- start 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 -- start start 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c102e90 0x7fe44c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c107570 0x7fe44c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c112530 con 0x7fe44c102e90 2024-04-24T22:40:51.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1123b0 con 0x7fe44c107570 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.528+0000 7fe452400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1126b0 con 0x7fe44c1024e0 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c102e90 0x7fe44c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45776/0 (socket says 172.21.15.111:45776) 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 -- 172.21.15.111:0/1267139305 learned_addr learned my addr 172.21.15.111:0/1267139305 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44be00700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c107570 0x7fe44c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 -- 172.21.15.111:0/1267139305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c107570 msgr2=0x7fe44c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c107570 0x7fe44c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 -- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c102e90 msgr2=0x7fe44c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c102e90 0x7fe44c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.529+0000 7fe44b400700 1 -- 172.21.15.111:0/1267139305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c198ff0 con 0x7fe44c1024e0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe44b400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9a0 tx=0x7fe43c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00ca10 con 0x7fe44c1024e0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c00cbb0 con 0x7fe44c1024e0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1992e0 con 0x7fe44c1024e0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c018680 con 0x7fe44c1024e0 2024-04-24T22:40:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.530+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c074460 con 0x7fe44c1024e0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.531+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c103970 con 0x7fe44c1024e0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.534+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe43c010430 con 0x7fe44c1024e0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.535+0000 7fe443e00700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 0x7fe4300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.535+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe43c0cf7f0 con 0x7fe44c1024e0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.535+0000 7fe44aa00700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 0x7fe4300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.536+0000 7fe44aa00700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 0x7fe4300a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:51.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.537+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe43c09c5a0 con 0x7fe44c1024e0 2024-04-24T22:40:51.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.788+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe44c02db30 con 0x7fe44c1024e0 2024-04-24T22:40:51.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.789+0000 7fe443e00700 1 -- 172.21.15.111:0/1267139305 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe43c09bea0 con 0x7fe44c1024e0 2024-04-24T22:40:51.791 INFO:teuthology.orchestra.run.smithi111.stdout:253403070511 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 msgr2=0x7fe4300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 0x7fe4300a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438005960 comp rx=0 tx=0).stop 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 msgr2=0x7fe44c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9a0 tx=0x7fe43c00ecb0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 shutdown_connections 2024-04-24T22:40:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fe4300a2cb0 0x7fe4300a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe44c107570 0x7fe44c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1024e0 0x7fe44c19e690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 --2- 172.21.15.111:0/1267139305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe44c102e90 0x7fe44c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 >> 172.21.15.111:0/1267139305 conn(0x7fe44c079390 msgr2=0x7fe44c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 shutdown_connections 2024-04-24T22:40:51.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:51.792+0000 7fe452400700 1 -- 172.21.15.111:0/1267139305 wait complete. 2024-04-24T22:40:52.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070511 for osd.8 2024-04-24T22:40:52.786 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:52.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:52 smithi171 ceph-mon[32337]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:52.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:52 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1267139305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T22:40:52.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:52 smithi111 ceph-mon[27746]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:52.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:52 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1267139305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T22:40:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:52 smithi186 ceph-mon[32753]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:52.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:52 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1267139305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T22:40:53.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 -- 172.21.15.111:0/2974234908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 msgr2=0x7fc298074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 --2- 172.21.15.111:0/2974234908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc298074840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc288009990 tx=0x7fc288009ca0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 -- 172.21.15.111:0/2974234908 shutdown_connections 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 --2- 172.21.15.111:0/2974234908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29810ec90 0x7fc298111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 --2- 172.21.15.111:0/2974234908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc298074840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 --2- 172.21.15.111:0/2974234908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc298074d80 0x7fc29810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 -- 172.21.15.111:0/2974234908 >> 172.21.15.111:0/2974234908 conn(0x7fc298079390 msgr2=0x7fc2980797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 -- 172.21.15.111:0/2974234908 shutdown_connections 2024-04-24T22:40:53.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.302+0000 7fc29ec00700 1 -- 172.21.15.111:0/2974234908 wait complete. 2024-04-24T22:40:53.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.303+0000 7fc29ec00700 1 Processor -- start 2024-04-24T22:40:53.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.303+0000 7fc29ec00700 1 -- start start 2024-04-24T22:40:53.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.303+0000 7fc29ec00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc2981a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29ec00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29ec00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981147e0 con 0x7fc29810ec90 2024-04-24T22:40:53.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29ec00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc298114660 con 0x7fc298074460 2024-04-24T22:40:53.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc298114960 con 0x7fc298074d80 2024-04-24T22:40:53.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29cc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc2981a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45788/0 (socket says 172.21.15.111:45788) 2024-04-24T22:40:53.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 -- 172.21.15.111:0/879229364 learned_addr learned my addr 172.21.15.111:0/879229364 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:53.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29cc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40184/0 (socket says 172.21.15.111:40184) 2024-04-24T22:40:53.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 -- 172.21.15.111:0/879229364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 msgr2=0x7fc2981a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc2981a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 -- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 msgr2=0x7fc29819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297400700 1 -- 172.21.15.111:0/879229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29819b140 con 0x7fc298074d80 2024-04-24T22:40:53.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc297e00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc2981a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:53.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.304+0000 7fc29cc00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:53.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc297400700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc28400eb30 tx=0x7fc28400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:53.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400cbd0 con 0x7fc298074d80 2024-04-24T22:40:53.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc284004500 con 0x7fc298074d80 2024-04-24T22:40:53.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29819b430 con 0x7fc298074d80 2024-04-24T22:40:53.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284010430 con 0x7fc298074d80 2024-04-24T22:40:53.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.305+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a76a0 con 0x7fc298074d80 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.306+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298068f20 con 0x7fc298074d80 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.309+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc284003f00 con 0x7fc298074d80 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.310+0000 7fc294c00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 0x7fc27c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.310+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc2840cf640 con 0x7fc298074d80 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.310+0000 7fc297e00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 0x7fc27c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.311+0000 7fc297e00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 0x7fc27c0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc298102200 tx=0x7fc288005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:53.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.312+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc28409c480 con 0x7fc298074d80 2024-04-24T22:40:53.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.540+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc29819bec0 con 0x7fc298074d80 2024-04-24T22:40:53.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.540+0000 7fc294c00700 1 -- 172.21.15.111:0/879229364 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc29819bec0 con 0x7fc298074d80 2024-04-24T22:40:53.542 INFO:teuthology.orchestra.run.smithi111.stdout:55834574935 2024-04-24T22:40:53.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.543+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 msgr2=0x7fc27c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.543+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 0x7fc27c0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc298102200 tx=0x7fc288005fd0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.543+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 msgr2=0x7fc2981a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.543+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc28400eb30 tx=0x7fc28400ee40 comp rx=0 tx=0).stop 2024-04-24T22:40:53.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 shutdown_connections 2024-04-24T22:40:53.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc29810ec90 0x7fc29819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc27c0a2cb0 0x7fc27c0a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc298074460 0x7fc2981a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 --2- 172.21.15.111:0/879229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc298074d80 0x7fc2981a0d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 >> 172.21.15.111:0/879229364 conn(0x7fc298079390 msgr2=0x7fc2980ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:53.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 shutdown_connections 2024-04-24T22:40:53.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.544+0000 7fc29ec00700 1 -- 172.21.15.111:0/879229364 wait complete. 2024-04-24T22:40:53.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 -- 172.21.15.111:0/353428763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c107570 msgr2=0x7fa40c10f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 --2- 172.21.15.111:0/353428763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c107570 0x7fa40c10f140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b0a0 tx=0x7fa40400b3b0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 -- 172.21.15.111:0/353428763 shutdown_connections 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 --2- 172.21.15.111:0/353428763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c107570 0x7fa40c10f140 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 --2- 172.21.15.111:0/353428763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 --2- 172.21.15.111:0/353428763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c102e90 0x7fa40c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.677+0000 7fa411000700 1 -- 172.21.15.111:0/353428763 >> 172.21.15.111:0/353428763 conn(0x7fa40c079390 msgr2=0x7fa40c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.678+0000 7fa411000700 1 -- 172.21.15.111:0/353428763 shutdown_connections 2024-04-24T22:40:53.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.678+0000 7fa411000700 1 -- 172.21.15.111:0/353428763 wait complete. 2024-04-24T22:40:53.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.679+0000 7fa411000700 1 Processor -- start 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.679+0000 7fa411000700 1 -- start start 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c102e90 0x7fa40c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 0x7fa40c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1125a0 con 0x7fa40c102e90 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112420 con 0x7fa40c107570 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa411000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112720 con 0x7fa40c1024e0 2024-04-24T22:40:53.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c102e90 0x7fa40c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45802/0 (socket says 172.21.15.111:45802) 2024-04-24T22:40:53.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 -- 172.21.15.111:0/46544788 learned_addr learned my addr 172.21.15.111:0/46544788 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:53.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa40a800700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 0x7fa40c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 -- 172.21.15.111:0/46544788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 msgr2=0x7fa40c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 0x7fa40c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 -- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c102e90 msgr2=0x7fa40c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c102e90 0x7fa40c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa409e00700 1 -- 172.21.15.111:0/46544788 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c19b180 con 0x7fa40c1024e0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.680+0000 7fa40a800700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 0x7fa40c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa409e00700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00cbd0 con 0x7fa40c1024e0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc00cd70 con 0x7fa40c1024e0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc010430 con 0x7fa40c1024e0 2024-04-24T22:40:53.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c19b470 con 0x7fa40c1024e0 2024-04-24T22:40:53.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.681+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a7730 con 0x7fa40c1024e0 2024-04-24T22:40:53.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.683+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c068f20 con 0x7fa40c1024e0 2024-04-24T22:40:53.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.684+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa3fc004750 con 0x7fa40c1024e0 2024-04-24T22:40:53.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.685+0000 7fa402a00700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 0x7fa3f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:53.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.686+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa3fc0cf400 con 0x7fa40c1024e0 2024-04-24T22:40:53.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.686+0000 7fa409400700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 0x7fa3f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:53.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.686+0000 7fa409400700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 0x7fa3f00a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa40c108d60 tx=0x7fa3f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:53.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.690+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3fc09c210 con 0x7fa40c1024e0 2024-04-24T22:40:53.926 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.924+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa40c1a7960 con 0x7fa40c1024e0 2024-04-24T22:40:53.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:53 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/879229364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T22:40:53.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.925+0000 7fa402a00700 1 -- 172.21.15.111:0/46544788 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa3fc09bb10 con 0x7fa40c1024e0 2024-04-24T22:40:53.927 INFO:teuthology.orchestra.run.smithi111.stdout:163208757313 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 msgr2=0x7fa3f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 0x7fa3f00a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa40c108d60 tx=0x7fa3f8009260 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 msgr2=0x7fa40c1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00ecd0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 shutdown_connections 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa3f00a2d30 0x7fa3f00a51f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40c107570 0x7fa40c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa40c1024e0 0x7fa40c1a0860 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 --2- 172.21.15.111:0/46544788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa40c102e90 0x7fa40c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:53.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 >> 172.21.15.111:0/46544788 conn(0x7fa40c079390 msgr2=0x7fa40c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:53.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 shutdown_connections 2024-04-24T22:40:53.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:53.929+0000 7fa411000700 1 -- 172.21.15.111:0/46544788 wait complete. 2024-04-24T22:40:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:53 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/879229364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T22:40:54.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:53 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/879229364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T22:40:54.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574935 for osd.1 2024-04-24T22:40:54.611 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:54.795 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:54 smithi111 ceph-mon[27746]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:54.795 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:54 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/46544788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T22:40:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:54 smithi186 ceph-mon[32753]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:54.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:54 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/46544788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T22:40:55.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:54 smithi171 ceph-mon[32337]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:40:55.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:54 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/46544788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T22:40:56.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757313 for osd.5 2024-04-24T22:40:56.117 DEBUG:teuthology.parallel:result is None 2024-04-24T22:40:56.117 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T22:40:56.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T22:40:56.569 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:40:56.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:56 smithi111 ceph-mon[27746]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:57.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:56 smithi186 ceph-mon[32753]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:57.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:56 smithi171 ceph-mon[32337]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:40:58.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 -- 172.21.15.111:0/1648365662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074460 msgr2=0x7feab8074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:58.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 --2- 172.21.15.111:0/1648365662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074460 0x7feab8074840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009f90 tx=0x7feaa800b330 comp rx=0 tx=0).stop 2024-04-24T22:40:58.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 -- 172.21.15.111:0/1648365662 shutdown_connections 2024-04-24T22:40:58.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 --2- 172.21.15.111:0/1648365662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab810ec30 0x7feab8111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:58.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 --2- 172.21.15.111:0/1648365662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074460 0x7feab8074840 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:58.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 --2- 172.21.15.111:0/1648365662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab8074d80 0x7feab810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:58.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 -- 172.21.15.111:0/1648365662 >> 172.21.15.111:0/1648365662 conn(0x7feab80793e0 msgr2=0x7feab80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:58.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 -- 172.21.15.111:0/1648365662 shutdown_connections 2024-04-24T22:40:58.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.886+0000 7feabde00700 1 -- 172.21.15.111:0/1648365662 wait complete. 2024-04-24T22:40:58.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:58 smithi111 ceph-mon[27746]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:58.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:58.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:58 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:58.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.887+0000 7feabde00700 1 Processor -- start 2024-04-24T22:40:58.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.887+0000 7feabde00700 1 -- start start 2024-04-24T22:40:58.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab8074460 0x7feab81a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:58.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074d80 0x7feab81a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:58.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feab81147a0 con 0x7feab810ec30 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feab8114620 con 0x7feab8074d80 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feabde00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feab8114920 con 0x7feab8074460 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feab7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feab7400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60296/0 (socket says 172.21.15.111:60296) 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feab6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab8074460 0x7feab81a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.888+0000 7feab7400700 1 -- 172.21.15.111:0/1330427539 learned_addr learned my addr 172.21.15.111:0/1330427539 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:40:58.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab6000700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074d80 0x7feab81a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 -- 172.21.15.111:0/1330427539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab8074460 msgr2=0x7feab81a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab8074460 0x7feab81a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 -- 172.21.15.111:0/1330427539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074d80 msgr2=0x7feab81a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074d80 0x7feab81a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 -- 172.21.15.111:0/1330427539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab819b1e0 con 0x7feab810ec30 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feab7400700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feab000e940 tx=0x7feab000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.889+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0004a30 con 0x7feab810ec30 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.890+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab000cca0 con 0x7feab810ec30 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.890+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0005610 con 0x7feab810ec30 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.890+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab819b4d0 con 0x7feab810ec30 2024-04-24T22:40:58.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.890+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab8102280 con 0x7feab810ec30 2024-04-24T22:40:58.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.891+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab8068f20 con 0x7feab810ec30 2024-04-24T22:40:58.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.893+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feab000c7c0 con 0x7feab810ec30 2024-04-24T22:40:58.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.895+0000 7feabca00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 0x7fea9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:58.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.896+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feab00ceff0 con 0x7feab810ec30 2024-04-24T22:40:58.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.896+0000 7feab6a00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 0x7fea9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:58.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.896+0000 7feab6a00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 0x7fea9c0a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009da0 tx=0x7feaa8009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:58.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:58.899+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feab009bf60 con 0x7feab810ec30 2024-04-24T22:40:58.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:58 smithi186 ceph-mon[32753]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:58 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:59.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.120+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feab80ffb50 con 0x7fea9c0a2ca0 2024-04-24T22:40:59.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.133+0000 7feabca00700 1 -- 172.21.15.111:0/1330427539 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258715 (secure 0 0 0) 0x7feab80ffb50 con 0x7fea9c0a2ca0 2024-04-24T22:40:59.136 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.140+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 msgr2=0x7fea9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 0x7fea9c0a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009da0 tx=0x7feaa8009720 comp rx=0 tx=0).stop 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 msgr2=0x7feab819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feab000e940 tx=0x7feab000ec50 comp rx=0 tx=0).stop 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 shutdown_connections 2024-04-24T22:40:59.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feab810ec30 0x7feab819aa80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab8074460 0x7feab81a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fea9c0a2ca0 0x7fea9c0a5160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 --2- 172.21.15.111:0/1330427539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feab8074d80 0x7feab81a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 >> 172.21.15.111:0/1330427539 conn(0x7feab80793e0 msgr2=0x7feab80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 shutdown_connections 2024-04-24T22:40:59.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:40:59.141+0000 7feabde00700 1 -- 172.21.15.111:0/1330427539 wait complete. 2024-04-24T22:40:59.145 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-04-24T22:40:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:58 smithi171 ceph-mon[32337]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:40:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:59.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:58 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:59.994 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:59 smithi111 ceph-mon[27746]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T22:40:59.994 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:59.994 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:59.995 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:40:59 smithi111 ceph-mon[27746]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:40:59.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:59 smithi186 ceph-mon[32753]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T22:41:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:40:59 smithi186 ceph-mon[32753]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:41:00.045 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-24T22:40:57.602818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654068,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":638,"num_read_kb":436,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379028,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562796,"statfs":{"total":1151940427776,"available":1151552303104,"internally_reserved":0,"allocated":13479936,"data_stored":7429584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704358+0000","last_change":"2024-04-24T22:39:30.580615+0000","last_active":"2024-04-24T22:40:28.704358+0000","last_peered":"2024-04-24T22:40:28.704358+0000","last_clean":"2024-04-24T22:40:28.704358+0000","last_became_active":"2024-04-24T22:39:30.580299+0000","last_became_peered":"2024-04-24T22:39:30.580299+0000","last_unstale":"2024-04-24T22:40:28.704358+0000","last_undegraded":"2024-04-24T22:40:28.704358+0000","last_fullsized":"2024-04-24T22:40:28.704358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.364115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706205+0000","last_change":"2024-04-24T22:39:30.588495+0000","last_active":"2024-04-24T22:40:28.706205+0000","last_peered":"2024-04-24T22:40:28.706205+0000","last_clean":"2024-04-24T22:40:28.706205+0000","last_became_active":"2024-04-24T22:39:30.585588+0000","last_became_peered":"2024-04-24T22:39:30.585588+0000","last_unstale":"2024-04-24T22:40:28.706205+0000","last_undegraded":"2024-04-24T22:40:28.706205+0000","last_fullsized":"2024-04-24T22:40:28.706205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.894832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707064+0000","last_change":"2024-04-24T22:39:30.585139+0000","last_active":"2024-04-24T22:40:28.707064+0000","last_peered":"2024-04-24T22:40:28.707064+0000","last_clean":"2024-04-24T22:40:28.707064+0000","last_became_active":"2024-04-24T22:39:30.583299+0000","last_became_peered":"2024-04-24T22:39:30.583299+0000","last_unstale":"2024-04-24T22:40:28.707064+0000","last_undegraded":"2024-04-24T22:40:28.707064+0000","last_fullsized":"2024-04-24T22:40:28.707064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:20.941383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.769187+0000","last_change":"2024-04-24T22:39:30.566540+0000","last_active":"2024-04-24T22:40:28.769187+0000","last_peered":"2024-04-24T22:40:28.769187+0000","last_clean":"2024-04-24T22:40:28.769187+0000","last_became_active":"2024-04-24T22:39:30.566195+0000","last_became_peered":"2024-04-24T22:39:30.566195+0000","last_unstale":"2024-04-24T22:40:28.769187+0000","last_undegraded":"2024-04-24T22:40:28.769187+0000","last_fullsized":"2024-04-24T22:40:28.769187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:24.538153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701839+0000","last_change":"2024-04-24T22:39:30.591842+0000","last_active":"2024-04-24T22:40:28.701839+0000","last_peered":"2024-04-24T22:40:28.701839+0000","last_clean":"2024-04-24T22:40:28.701839+0000","last_became_active":"2024-04-24T22:39:30.591579+0000","last_became_peered":"2024-04-24T22:39:30.591579+0000","last_unstale":"2024-04-24T22:40:28.701839+0000","last_undegraded":"2024-04-24T22:40:28.701839+0000","last_fullsized":"2024-04-24T22:40:28.701839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:48.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707226+0000","last_change":"2024-04-24T22:39:30.577689+0000","last_active":"2024-04-24T22:40:28.707226+0000","last_peered":"2024-04-24T22:40:28.707226+0000","last_clean":"2024-04-24T22:40:28.707226+0000","last_became_active":"2024-04-24T22:39:30.577362+0000","last_became_peered":"2024-04-24T22:39:30.577362+0000","last_unstale":"2024-04-24T22:40:28.707226+0000","last_undegraded":"2024-04-24T22:40:28.707226+0000","last_fullsized":"2024-04-24T22:40:28.707226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:52.808898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701687+0000","last_change":"2024-04-24T22:39:30.576400+0000","last_active":"2024-04-24T22:40:28.701687+0000","last_peered":"2024-04-24T22:40:28.701687+0000","last_clean":"2024-04-24T22:40:28.701687+0000","last_became_active":"2024-04-24T22:39:30.576141+0000","last_became_peered":"2024-04-24T22:39:30.576141+0000","last_unstale":"2024-04-24T22:40:28.701687+0000","last_undegraded":"2024-04-24T22:40:28.701687+0000","last_fullsized":"2024-04-24T22:40:28.701687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.710279+0000","last_change":"2024-04-24T22:39:30.594496+0000","last_active":"2024-04-24T22:40:28.710279+0000","last_peered":"2024-04-24T22:40:28.710279+0000","last_clean":"2024-04-24T22:40:28.710279+0000","last_became_active":"2024-04-24T22:39:30.589541+0000","last_became_peered":"2024-04-24T22:39:30.589541+0000","last_unstale":"2024-04-24T22:40:28.710279+0000","last_undegraded":"2024-04-24T22:40:28.710279+0000","last_fullsized":"2024-04-24T22:40:28.710279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:50.593142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.924155+0000","last_change":"2024-04-24T22:39:30.567922+0000","last_active":"2024-04-24T22:40:28.924155+0000","last_peered":"2024-04-24T22:40:28.924155+0000","last_clean":"2024-04-24T22:40:28.924155+0000","last_became_active":"2024-04-24T22:39:30.567470+0000","last_became_peered":"2024-04-24T22:39:30.567470+0000","last_unstale":"2024-04-24T22:40:28.924155+0000","last_undegraded":"2024-04-24T22:40:28.924155+0000","last_fullsized":"2024-04-24T22:40:28.924155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:11.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.152804+0000","last_change":"2024-04-24T22:40:29.152804+0000","last_active":"2024-04-24T22:40:29.152804+0000","last_peered":"2024-04-24T22:40:29.152804+0000","last_clean":"2024-04-24T22:40:29.152804+0000","last_became_active":"2024-04-24T22:40:29.152234+0000","last_became_peered":"2024-04-24T22:40:29.152234+0000","last_unstale":"2024-04-24T22:40:29.152804+0000","last_undegraded":"2024-04-24T22:40:29.152804+0000","last_fullsized":"2024-04-24T22:40:29.152804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:39.167469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702237+0000","last_change":"2024-04-24T22:39:30.571679+0000","last_active":"2024-04-24T22:40:28.702237+0000","last_peered":"2024-04-24T22:40:28.702237+0000","last_clean":"2024-04-24T22:40:28.702237+0000","last_became_active":"2024-04-24T22:39:30.571375+0000","last_became_peered":"2024-04-24T22:39:30.571375+0000","last_unstale":"2024-04-24T22:40:28.702237+0000","last_undegraded":"2024-04-24T22:40:28.702237+0000","last_fullsized":"2024-04-24T22:40:28.702237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:47.219082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.768974+0000","last_change":"2024-04-24T22:39:30.565158+0000","last_active":"2024-04-24T22:40:28.768974+0000","last_peered":"2024-04-24T22:40:28.768974+0000","last_clean":"2024-04-24T22:40:28.768974+0000","last_became_active":"2024-04-24T22:39:30.564808+0000","last_became_peered":"2024-04-24T22:39:30.564808+0000","last_unstale":"2024-04-24T22:40:28.768974+0000","last_undegraded":"2024-04-24T22:40:28.768974+0000","last_fullsized":"2024-04-24T22:40:28.768974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:33.534801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.710881+0000","last_change":"2024-04-24T22:39:30.561125+0000","last_active":"2024-04-24T22:40:28.710881+0000","last_peered":"2024-04-24T22:40:28.710881+0000","last_clean":"2024-04-24T22:40:28.710881+0000","last_became_active":"2024-04-24T22:39:30.559473+0000","last_became_peered":"2024-04-24T22:39:30.559473+0000","last_unstale":"2024-04-24T22:40:28.710881+0000","last_undegraded":"2024-04-24T22:40:28.710881+0000","last_fullsized":"2024-04-24T22:40:28.710881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:09.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704446+0000","last_change":"2024-04-24T22:39:30.582371+0000","last_active":"2024-04-24T22:40:28.704446+0000","last_peered":"2024-04-24T22:40:28.704446+0000","last_clean":"2024-04-24T22:40:28.704446+0000","last_became_active":"2024-04-24T22:39:30.582002+0000","last_became_peered":"2024-04-24T22:39:30.582002+0000","last_unstale":"2024-04-24T22:40:28.704446+0000","last_undegraded":"2024-04-24T22:40:28.704446+0000","last_fullsized":"2024-04-24T22:40:28.704446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:11.812461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.716114+0000","last_change":"2024-04-24T22:40:28.716114+0000","last_active":"2024-04-24T22:40:28.716114+0000","last_peered":"2024-04-24T22:40:28.716114+0000","last_clean":"2024-04-24T22:40:28.716114+0000","last_became_active":"2024-04-24T22:40:28.715344+0000","last_became_peered":"2024-04-24T22:40:28.715344+0000","last_unstale":"2024-04-24T22:40:28.716114+0000","last_undegraded":"2024-04-24T22:40:28.716114+0000","last_fullsized":"2024-04-24T22:40:28.716114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:31.943912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923734+0000","last_change":"2024-04-24T22:39:30.578310+0000","last_active":"2024-04-24T22:40:28.923734+0000","last_peered":"2024-04-24T22:40:28.923734+0000","last_clean":"2024-04-24T22:40:28.923734+0000","last_became_active":"2024-04-24T22:39:30.577917+0000","last_became_peered":"2024-04-24T22:39:30.577917+0000","last_unstale":"2024-04-24T22:40:28.923734+0000","last_undegraded":"2024-04-24T22:40:28.923734+0000","last_fullsized":"2024-04-24T22:40:28.923734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:51.766196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705817+0000","last_change":"2024-04-24T22:39:30.580939+0000","last_active":"2024-04-24T22:40:28.705817+0000","last_peered":"2024-04-24T22:40:28.705817+0000","last_clean":"2024-04-24T22:40:28.705817+0000","last_became_active":"2024-04-24T22:39:30.580583+0000","last_became_peered":"2024-04-24T22:39:30.580583+0000","last_unstale":"2024-04-24T22:40:28.705817+0000","last_undegraded":"2024-04-24T22:40:28.705817+0000","last_fullsized":"2024-04-24T22:40:28.705817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.891103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.150746+0000","last_change":"2024-04-24T22:39:30.577933+0000","last_active":"2024-04-24T22:40:29.150746+0000","last_peered":"2024-04-24T22:40:29.150746+0000","last_clean":"2024-04-24T22:40:29.150746+0000","last_became_active":"2024-04-24T22:39:30.577545+0000","last_became_peered":"2024-04-24T22:39:30.577545+0000","last_unstale":"2024-04-24T22:40:29.150746+0000","last_undegraded":"2024-04-24T22:40:29.150746+0000","last_fullsized":"2024-04-24T22:40:29.150746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:26.876281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706673+0000","last_change":"2024-04-24T22:39:30.586170+0000","last_active":"2024-04-24T22:40:28.706673+0000","last_peered":"2024-04-24T22:40:28.706673+0000","last_clean":"2024-04-24T22:40:28.706673+0000","last_became_active":"2024-04-24T22:39:30.585922+0000","last_became_peered":"2024-04-24T22:39:30.585922+0000","last_unstale":"2024-04-24T22:40:28.706673+0000","last_undegraded":"2024-04-24T22:40:28.706673+0000","last_fullsized":"2024-04-24T22:40:28.706673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:53.166725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.150655+0000","last_change":"2024-04-24T22:39:29.660103+0000","last_active":"2024-04-24T22:40:29.150655+0000","last_peered":"2024-04-24T22:40:29.150655+0000","last_clean":"2024-04-24T22:40:29.150655+0000","last_became_active":"2024-04-24T22:39:29.659809+0000","last_became_peered":"2024-04-24T22:39:29.659809+0000","last_unstale":"2024-04-24T22:40:29.150655+0000","last_undegraded":"2024-04-24T22:40:29.150655+0000","last_fullsized":"2024-04-24T22:40:29.150655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:40.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.714694+0000","last_change":"2024-04-24T22:39:30.587041+0000","last_active":"2024-04-24T22:40:28.714694+0000","last_peered":"2024-04-24T22:40:28.714694+0000","last_clean":"2024-04-24T22:40:28.714694+0000","last_became_active":"2024-04-24T22:39:30.586568+0000","last_became_peered":"2024-04-24T22:39:30.586568+0000","last_unstale":"2024-04-24T22:40:28.714694+0000","last_undegraded":"2024-04-24T22:40:28.714694+0000","last_fullsized":"2024-04-24T22:40:28.714694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:13.305790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.715537+0000","last_change":"2024-04-24T22:39:29.659591+0000","last_active":"2024-04-24T22:40:28.715537+0000","last_peered":"2024-04-24T22:40:28.715537+0000","last_clean":"2024-04-24T22:40:28.715537+0000","last_became_active":"2024-04-24T22:39:29.659281+0000","last_became_peered":"2024-04-24T22:39:29.659281+0000","last_unstale":"2024-04-24T22:40:28.715537+0000","last_undegraded":"2024-04-24T22:40:28.715537+0000","last_fullsized":"2024-04-24T22:40:28.715537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:07.909300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.708559+0000","last_change":"2024-04-24T22:39:30.589041+0000","last_active":"2024-04-24T22:40:28.708559+0000","last_peered":"2024-04-24T22:40:28.708559+0000","last_clean":"2024-04-24T22:40:28.708559+0000","last_became_active":"2024-04-24T22:39:30.588706+0000","last_became_peered":"2024-04-24T22:39:30.588706+0000","last_unstale":"2024-04-24T22:40:28.708559+0000","last_undegraded":"2024-04-24T22:40:28.708559+0000","last_fullsized":"2024-04-24T22:40:28.708559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:19.594236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702327+0000","last_change":"2024-04-24T22:39:29.659280+0000","last_active":"2024-04-24T22:40:28.702327+0000","last_peered":"2024-04-24T22:40:28.702327+0000","last_clean":"2024-04-24T22:40:28.702327+0000","last_became_active":"2024-04-24T22:39:29.658922+0000","last_became_peered":"2024-04-24T22:39:29.658922+0000","last_unstale":"2024-04-24T22:40:28.702327+0000","last_undegraded":"2024-04-24T22:40:28.702327+0000","last_fullsized":"2024-04-24T22:40:28.702327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:58.157675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923943+0000","last_change":"2024-04-24T22:39:30.573769+0000","last_active":"2024-04-24T22:40:28.923943+0000","last_peered":"2024-04-24T22:40:28.923943+0000","last_clean":"2024-04-24T22:40:28.923943+0000","last_became_active":"2024-04-24T22:39:30.573405+0000","last_became_peered":"2024-04-24T22:39:30.573405+0000","last_unstale":"2024-04-24T22:40:28.923943+0000","last_undegraded":"2024-04-24T22:40:28.923943+0000","last_fullsized":"2024-04-24T22:40:28.923943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:08.919808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702136+0000","last_change":"2024-04-24T22:39:29.653759+0000","last_active":"2024-04-24T22:40:28.702136+0000","last_peered":"2024-04-24T22:40:28.702136+0000","last_clean":"2024-04-24T22:40:28.702136+0000","last_became_active":"2024-04-24T22:39:29.653060+0000","last_became_peered":"2024-04-24T22:39:29.653060+0000","last_unstale":"2024-04-24T22:40:28.702136+0000","last_undegraded":"2024-04-24T22:40:28.702136+0000","last_fullsized":"2024-04-24T22:40:28.702136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:02.503014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078237+0000","last_change":"2024-04-24T22:39:30.563070+0000","last_active":"2024-04-24T22:40:29.078237+0000","last_peered":"2024-04-24T22:40:29.078237+0000","last_clean":"2024-04-24T22:40:29.078237+0000","last_became_active":"2024-04-24T22:39:30.562736+0000","last_became_peered":"2024-04-24T22:39:30.562736+0000","last_unstale":"2024-04-24T22:40:29.078237+0000","last_undegraded":"2024-04-24T22:40:29.078237+0000","last_fullsized":"2024-04-24T22:40:29.078237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:15.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:55.114254+0000","last_change":"2024-04-24T22:39:29.657659+0000","last_active":"2024-04-24T22:40:55.114254+0000","last_peered":"2024-04-24T22:40:55.114254+0000","last_clean":"2024-04-24T22:40:55.114254+0000","last_became_active":"2024-04-24T22:39:29.657237+0000","last_became_peered":"2024-04-24T22:39:29.657237+0000","last_unstale":"2024-04-24T22:40:55.114254+0000","last_undegraded":"2024-04-24T22:40:55.114254+0000","last_fullsized":"2024-04-24T22:40:55.114254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:11.376959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701898+0000","last_change":"2024-04-24T22:39:30.577542+0000","last_active":"2024-04-24T22:40:28.701898+0000","last_peered":"2024-04-24T22:40:28.701898+0000","last_clean":"2024-04-24T22:40:28.701898+0000","last_became_active":"2024-04-24T22:39:30.577194+0000","last_became_peered":"2024-04-24T22:39:30.577194+0000","last_unstale":"2024-04-24T22:40:28.701898+0000","last_undegraded":"2024-04-24T22:40:28.701898+0000","last_fullsized":"2024-04-24T22:40:28.701898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:26.000787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707141+0000","last_change":"2024-04-24T22:39:29.671709+0000","last_active":"2024-04-24T22:40:28.707141+0000","last_peered":"2024-04-24T22:40:28.707141+0000","last_clean":"2024-04-24T22:40:28.707141+0000","last_became_active":"2024-04-24T22:39:29.671306+0000","last_became_peered":"2024-04-24T22:39:29.671306+0000","last_unstale":"2024-04-24T22:40:28.707141+0000","last_undegraded":"2024-04-24T22:40:28.707141+0000","last_fullsized":"2024-04-24T22:40:28.707141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:33.833217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078035+0000","last_change":"2024-04-24T22:39:30.583720+0000","last_active":"2024-04-24T22:40:29.078035+0000","last_peered":"2024-04-24T22:40:29.078035+0000","last_clean":"2024-04-24T22:40:29.078035+0000","last_became_active":"2024-04-24T22:39:30.583468+0000","last_became_peered":"2024-04-24T22:39:30.583468+0000","last_unstale":"2024-04-24T22:40:29.078035+0000","last_undegraded":"2024-04-24T22:40:29.078035+0000","last_fullsized":"2024-04-24T22:40:29.078035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:07.679284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.709760+0000","last_change":"2024-04-24T22:39:29.669100+0000","last_active":"2024-04-24T22:40:28.709760+0000","last_peered":"2024-04-24T22:40:28.709760+0000","last_clean":"2024-04-24T22:40:28.709760+0000","last_became_active":"2024-04-24T22:39:29.668773+0000","last_became_peered":"2024-04-24T22:39:29.668773+0000","last_unstale":"2024-04-24T22:40:28.709760+0000","last_undegraded":"2024-04-24T22:40:28.709760+0000","last_fullsized":"2024-04-24T22:40:28.709760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:56.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.711009+0000","last_change":"2024-04-24T22:39:30.594562+0000","last_active":"2024-04-24T22:40:28.711009+0000","last_peered":"2024-04-24T22:40:28.711009+0000","last_clean":"2024-04-24T22:40:28.711009+0000","last_became_active":"2024-04-24T22:39:30.589928+0000","last_became_peered":"2024-04-24T22:39:30.589928+0000","last_unstale":"2024-04-24T22:40:28.711009+0000","last_undegraded":"2024-04-24T22:40:28.711009+0000","last_fullsized":"2024-04-24T22:40:28.711009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:14.951690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077769+0000","last_change":"2024-04-24T22:39:29.658615+0000","last_active":"2024-04-24T22:40:29.077769+0000","last_peered":"2024-04-24T22:40:29.077769+0000","last_clean":"2024-04-24T22:40:29.077769+0000","last_became_active":"2024-04-24T22:39:29.658178+0000","last_became_peered":"2024-04-24T22:39:29.658178+0000","last_unstale":"2024-04-24T22:40:29.077769+0000","last_undegraded":"2024-04-24T22:40:29.077769+0000","last_fullsized":"2024-04-24T22:40:29.077769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.821509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.149667+0000","last_change":"2024-04-24T22:39:30.592142+0000","last_active":"2024-04-24T22:40:29.149667+0000","last_peered":"2024-04-24T22:40:29.149667+0000","last_clean":"2024-04-24T22:40:29.149667+0000","last_became_active":"2024-04-24T22:39:30.591417+0000","last_became_peered":"2024-04-24T22:39:30.591417+0000","last_unstale":"2024-04-24T22:40:29.149667+0000","last_undegraded":"2024-04-24T22:40:29.149667+0000","last_fullsized":"2024-04-24T22:40:29.149667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:55.818795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:55.113593+0000","last_change":"2024-04-24T22:39:29.662513+0000","last_active":"2024-04-24T22:40:55.113593+0000","last_peered":"2024-04-24T22:40:55.113593+0000","last_clean":"2024-04-24T22:40:55.113593+0000","last_became_active":"2024-04-24T22:39:29.662223+0000","last_became_peered":"2024-04-24T22:39:29.662223+0000","last_unstale":"2024-04-24T22:40:55.113593+0000","last_undegraded":"2024-04-24T22:40:55.113593+0000","last_fullsized":"2024-04-24T22:40:55.113593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.012595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.924088+0000","last_change":"2024-04-24T22:39:30.577594+0000","last_active":"2024-04-24T22:40:28.924088+0000","last_peered":"2024-04-24T22:40:28.924088+0000","last_clean":"2024-04-24T22:40:28.924088+0000","last_became_active":"2024-04-24T22:39:30.577254+0000","last_became_peered":"2024-04-24T22:39:30.577254+0000","last_unstale":"2024-04-24T22:40:28.924088+0000","last_undegraded":"2024-04-24T22:40:28.924088+0000","last_fullsized":"2024-04-24T22:40:28.924088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:47.926878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701897+0000","last_change":"2024-04-24T22:39:29.659898+0000","last_active":"2024-04-24T22:40:28.701897+0000","last_peered":"2024-04-24T22:40:28.701897+0000","last_clean":"2024-04-24T22:40:28.701897+0000","last_became_active":"2024-04-24T22:39:29.659582+0000","last_became_peered":"2024-04-24T22:39:29.659582+0000","last_unstale":"2024-04-24T22:40:28.701897+0000","last_undegraded":"2024-04-24T22:40:28.701897+0000","last_fullsized":"2024-04-24T22:40:28.701897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.708291+0000","last_change":"2024-04-24T22:39:30.588271+0000","last_active":"2024-04-24T22:40:28.708291+0000","last_peered":"2024-04-24T22:40:28.708291+0000","last_clean":"2024-04-24T22:40:28.708291+0000","last_became_active":"2024-04-24T22:39:30.578846+0000","last_became_peered":"2024-04-24T22:39:30.578846+0000","last_unstale":"2024-04-24T22:40:28.708291+0000","last_undegraded":"2024-04-24T22:40:28.708291+0000","last_fullsized":"2024-04-24T22:40:28.708291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:23.807550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078249+0000","last_change":"2024-04-24T22:39:29.654708+0000","last_active":"2024-04-24T22:40:29.078249+0000","last_peered":"2024-04-24T22:40:29.078249+0000","last_clean":"2024-04-24T22:40:29.078249+0000","last_became_active":"2024-04-24T22:39:29.654290+0000","last_became_peered":"2024-04-24T22:39:29.654290+0000","last_unstale":"2024-04-24T22:40:29.078249+0000","last_undegraded":"2024-04-24T22:40:29.078249+0000","last_fullsized":"2024-04-24T22:40:29.078249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:29.445700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923934+0000","last_change":"2024-04-24T22:39:30.569436+0000","last_active":"2024-04-24T22:40:28.923934+0000","last_peered":"2024-04-24T22:40:28.923934+0000","last_clean":"2024-04-24T22:40:28.923934+0000","last_became_active":"2024-04-24T22:39:30.569104+0000","last_became_peered":"2024-04-24T22:39:30.569104+0000","last_unstale":"2024-04-24T22:40:28.923934+0000","last_undegraded":"2024-04-24T22:40:28.923934+0000","last_fullsized":"2024-04-24T22:40:28.923934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:30.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":215,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:56.114777+0000","last_change":"2024-04-24T22:39:29.657883+0000","last_active":"2024-04-24T22:40:56.114777+0000","last_peered":"2024-04-24T22:40:56.114777+0000","last_clean":"2024-04-24T22:40:56.114777+0000","last_became_active":"2024-04-24T22:39:29.657496+0000","last_became_peered":"2024-04-24T22:39:29.657496+0000","last_unstale":"2024-04-24T22:40:56.114777+0000","last_undegraded":"2024-04-24T22:40:56.114777+0000","last_fullsized":"2024-04-24T22:40:56.114777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:51:32.037937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":169,"num_read_kb":142,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702206+0000","last_change":"2024-04-24T22:39:30.569744+0000","last_active":"2024-04-24T22:40:28.702206+0000","last_peered":"2024-04-24T22:40:28.702206+0000","last_clean":"2024-04-24T22:40:28.702206+0000","last_became_active":"2024-04-24T22:39:30.569392+0000","last_became_peered":"2024-04-24T22:39:30.569392+0000","last_unstale":"2024-04-24T22:40:28.702206+0000","last_undegraded":"2024-04-24T22:40:28.702206+0000","last_fullsized":"2024-04-24T22:40:28.702206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:09.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.725538+0000","last_change":"2024-04-24T22:40:28.725141+0000","last_active":"2024-04-24T22:40:28.725538+0000","last_peered":"2024-04-24T22:40:28.725538+0000","last_clean":"2024-04-24T22:40:28.725538+0000","last_became_active":"2024-04-24T22:40:28.713510+0000","last_became_peered":"2024-04-24T22:40:28.713510+0000","last_unstale":"2024-04-24T22:40:28.725538+0000","last_undegraded":"2024-04-24T22:40:28.725538+0000","last_fullsized":"2024-04-24T22:40:28.725538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:09.796454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.153608+0000","last_change":"2024-04-24T22:40:29.153608+0000","last_active":"2024-04-24T22:40:29.153608+0000","last_peered":"2024-04-24T22:40:29.153608+0000","last_clean":"2024-04-24T22:40:29.153608+0000","last_became_active":"2024-04-24T22:40:29.152328+0000","last_became_peered":"2024-04-24T22:40:29.152328+0000","last_unstale":"2024-04-24T22:40:29.153608+0000","last_undegraded":"2024-04-24T22:40:29.153608+0000","last_fullsized":"2024-04-24T22:40:29.153608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:15.265528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077873+0000","last_change":"2024-04-24T22:39:29.657561+0000","last_active":"2024-04-24T22:40:29.077873+0000","last_peered":"2024-04-24T22:40:29.077873+0000","last_clean":"2024-04-24T22:40:29.077873+0000","last_became_active":"2024-04-24T22:39:29.657261+0000","last_became_peered":"2024-04-24T22:39:29.657261+0000","last_unstale":"2024-04-24T22:40:29.077873+0000","last_undegraded":"2024-04-24T22:40:29.077873+0000","last_fullsized":"2024-04-24T22:40:29.077873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:11.856640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701777+0000","last_change":"2024-04-24T22:39:30.577405+0000","last_active":"2024-04-24T22:40:28.701777+0000","last_peered":"2024-04-24T22:40:28.701777+0000","last_clean":"2024-04-24T22:40:28.701777+0000","last_became_active":"2024-04-24T22:39:30.576980+0000","last_became_peered":"2024-04-24T22:39:30.576980+0000","last_unstale":"2024-04-24T22:40:28.701777+0000","last_undegraded":"2024-04-24T22:40:28.701777+0000","last_fullsized":"2024-04-24T22:40:28.701777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:00.145497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:51.169749+0000","last_change":"2024-04-24T22:39:29.661054+0000","last_active":"2024-04-24T22:40:51.169749+0000","last_peered":"2024-04-24T22:40:51.169749+0000","last_clean":"2024-04-24T22:40:51.169749+0000","last_became_active":"2024-04-24T22:39:29.660638+0000","last_became_peered":"2024-04-24T22:39:29.660638+0000","last_unstale":"2024-04-24T22:40:51.169749+0000","last_undegraded":"2024-04-24T22:40:51.169749+0000","last_fullsized":"2024-04-24T22:40:51.169749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:51.891654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701595+0000","last_change":"2024-04-24T22:39:30.570407+0000","last_active":"2024-04-24T22:40:28.701595+0000","last_peered":"2024-04-24T22:40:28.701595+0000","last_clean":"2024-04-24T22:40:28.701595+0000","last_became_active":"2024-04-24T22:39:30.570070+0000","last_became_peered":"2024-04-24T22:39:30.570070+0000","last_unstale":"2024-04-24T22:40:28.701595+0000","last_undegraded":"2024-04-24T22:40:28.701595+0000","last_fullsized":"2024-04-24T22:40:28.701595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.785122+0000","last_change":"2024-04-24T22:40:28.785122+0000","last_active":"2024-04-24T22:40:28.785122+0000","last_peered":"2024-04-24T22:40:28.785122+0000","last_clean":"2024-04-24T22:40:28.785122+0000","last_became_active":"2024-04-24T22:40:28.770463+0000","last_became_peered":"2024-04-24T22:40:28.770463+0000","last_unstale":"2024-04-24T22:40:28.785122+0000","last_undegraded":"2024-04-24T22:40:28.785122+0000","last_fullsized":"2024-04-24T22:40:28.785122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:18.755290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077842+0000","last_change":"2024-04-24T22:39:30.562092+0000","last_active":"2024-04-24T22:40:29.077842+0000","last_peered":"2024-04-24T22:40:29.077842+0000","last_clean":"2024-04-24T22:40:29.077842+0000","last_became_active":"2024-04-24T22:39:30.561704+0000","last_became_peered":"2024-04-24T22:39:30.561704+0000","last_unstale":"2024-04-24T22:40:29.077842+0000","last_undegraded":"2024-04-24T22:40:29.077842+0000","last_fullsized":"2024-04-24T22:40:29.077842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.996837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702283+0000","last_change":"2024-04-24T22:39:29.655612+0000","last_active":"2024-04-24T22:40:28.702283+0000","last_peered":"2024-04-24T22:40:28.702283+0000","last_clean":"2024-04-24T22:40:28.702283+0000","last_became_active":"2024-04-24T22:39:29.655212+0000","last_became_peered":"2024-04-24T22:39:29.655212+0000","last_unstale":"2024-04-24T22:40:28.702283+0000","last_undegraded":"2024-04-24T22:40:28.702283+0000","last_fullsized":"2024-04-24T22:40:28.702283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.414774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.152885+0000","last_change":"2024-04-24T22:40:29.152885+0000","last_active":"2024-04-24T22:40:29.152885+0000","last_peered":"2024-04-24T22:40:29.152885+0000","last_clean":"2024-04-24T22:40:29.152885+0000","last_became_active":"2024-04-24T22:40:29.152490+0000","last_became_peered":"2024-04-24T22:40:29.152490+0000","last_unstale":"2024-04-24T22:40:29.152885+0000","last_undegraded":"2024-04-24T22:40:29.152885+0000","last_fullsized":"2024-04-24T22:40:29.152885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:54.660889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.150415+0000","last_change":"2024-04-24T22:39:29.661052+0000","last_active":"2024-04-24T22:40:29.150415+0000","last_peered":"2024-04-24T22:40:29.150415+0000","last_clean":"2024-04-24T22:40:29.150415+0000","last_became_active":"2024-04-24T22:39:29.660716+0000","last_became_peered":"2024-04-24T22:39:29.660716+0000","last_unstale":"2024-04-24T22:40:29.150415+0000","last_undegraded":"2024-04-24T22:40:29.150415+0000","last_fullsized":"2024-04-24T22:40:29.150415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:52.016503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707160+0000","last_change":"2024-04-24T22:39:30.585190+0000","last_active":"2024-04-24T22:40:28.707160+0000","last_peered":"2024-04-24T22:40:28.707160+0000","last_clean":"2024-04-24T22:40:28.707160+0000","last_became_active":"2024-04-24T22:39:30.584846+0000","last_became_peered":"2024-04-24T22:39:30.584846+0000","last_unstale":"2024-04-24T22:40:28.707160+0000","last_undegraded":"2024-04-24T22:40:28.707160+0000","last_fullsized":"2024-04-24T22:40:28.707160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:07.094297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701979+0000","last_change":"2024-04-24T22:39:29.671686+0000","last_active":"2024-04-24T22:40:28.701979+0000","last_peered":"2024-04-24T22:40:28.701979+0000","last_clean":"2024-04-24T22:40:28.701979+0000","last_became_active":"2024-04-24T22:39:29.671338+0000","last_became_peered":"2024-04-24T22:39:29.671338+0000","last_unstale":"2024-04-24T22:40:28.701979+0000","last_undegraded":"2024-04-24T22:40:28.701979+0000","last_fullsized":"2024-04-24T22:40:28.701979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:13.709429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707163+0000","last_change":"2024-04-24T22:39:30.585560+0000","last_active":"2024-04-24T22:40:28.707163+0000","last_peered":"2024-04-24T22:40:28.707163+0000","last_clean":"2024-04-24T22:40:28.707163+0000","last_became_active":"2024-04-24T22:39:30.585251+0000","last_became_peered":"2024-04-24T22:39:30.585251+0000","last_unstale":"2024-04-24T22:40:28.707163+0000","last_undegraded":"2024-04-24T22:40:28.707163+0000","last_fullsized":"2024-04-24T22:40:28.707163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:29.680556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077985+0000","last_change":"2024-04-24T22:39:30.569105+0000","last_active":"2024-04-24T22:40:29.077985+0000","last_peered":"2024-04-24T22:40:29.077985+0000","last_clean":"2024-04-24T22:40:29.077985+0000","last_became_active":"2024-04-24T22:39:30.566878+0000","last_became_peered":"2024-04-24T22:39:30.566878+0000","last_unstale":"2024-04-24T22:40:29.077985+0000","last_undegraded":"2024-04-24T22:40:29.077985+0000","last_fullsized":"2024-04-24T22:40:29.077985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.334294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:51.168938+0000","last_change":"2024-04-24T22:39:29.674334+0000","last_active":"2024-04-24T22:40:51.168938+0000","last_peered":"2024-04-24T22:40:51.168938+0000","last_clean":"2024-04-24T22:40:51.168938+0000","last_became_active":"2024-04-24T22:39:29.673968+0000","last_became_peered":"2024-04-24T22:39:29.673968+0000","last_unstale":"2024-04-24T22:40:51.168938+0000","last_undegraded":"2024-04-24T22:40:51.168938+0000","last_fullsized":"2024-04-24T22:40:51.168938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:10.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.713359+0000","last_change":"2024-04-24T22:39:30.579663+0000","last_active":"2024-04-24T22:40:28.713359+0000","last_peered":"2024-04-24T22:40:28.713359+0000","last_clean":"2024-04-24T22:40:28.713359+0000","last_became_active":"2024-04-24T22:39:30.578730+0000","last_became_peered":"2024-04-24T22:39:30.578730+0000","last_unstale":"2024-04-24T22:40:28.713359+0000","last_undegraded":"2024-04-24T22:40:28.713359+0000","last_fullsized":"2024-04-24T22:40:28.713359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:25.604081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.768758+0000","last_change":"2024-04-24T22:39:29.655316+0000","last_active":"2024-04-24T22:40:28.768758+0000","last_peered":"2024-04-24T22:40:28.768758+0000","last_clean":"2024-04-24T22:40:28.768758+0000","last_became_active":"2024-04-24T22:39:29.654975+0000","last_became_peered":"2024-04-24T22:39:29.654975+0000","last_unstale":"2024-04-24T22:40:28.768758+0000","last_undegraded":"2024-04-24T22:40:28.768758+0000","last_fullsized":"2024-04-24T22:40:28.768758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:20.461744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701776+0000","last_change":"2024-04-24T22:38:09.915368+0000","last_active":"2024-04-24T22:40:28.701776+0000","last_peered":"2024-04-24T22:40:28.701776+0000","last_clean":"2024-04-24T22:40:28.701776+0000","last_became_active":"2024-04-24T22:38:09.914879+0000","last_became_peered":"2024-04-24T22:38:09.914879+0000","last_unstale":"2024-04-24T22:40:28.701776+0000","last_undegraded":"2024-04-24T22:40:28.701776+0000","last_fullsized":"2024-04-24T22:40:28.701776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_clean_scrub_stamp":"2024-04-24T22:34:20.136986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:31.054930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.709178+0000","last_change":"2024-04-24T22:39:30.568169+0000","last_active":"2024-04-24T22:40:28.709178+0000","last_peered":"2024-04-24T22:40:28.709178+0000","last_clean":"2024-04-24T22:40:28.709178+0000","last_became_active":"2024-04-24T22:39:30.567221+0000","last_became_peered":"2024-04-24T22:39:30.567221+0000","last_unstale":"2024-04-24T22:40:28.709178+0000","last_undegraded":"2024-04-24T22:40:28.709178+0000","last_fullsized":"2024-04-24T22:40:28.709178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:06.414726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701859+0000","last_change":"2024-04-24T22:39:29.660378+0000","last_active":"2024-04-24T22:40:28.701859+0000","last_peered":"2024-04-24T22:40:28.701859+0000","last_clean":"2024-04-24T22:40:28.701859+0000","last_became_active":"2024-04-24T22:39:29.660050+0000","last_became_peered":"2024-04-24T22:39:29.660050+0000","last_unstale":"2024-04-24T22:40:28.701859+0000","last_undegraded":"2024-04-24T22:40:28.701859+0000","last_fullsized":"2024-04-24T22:40:28.701859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:06.199365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702124+0000","last_change":"2024-04-24T22:39:29.674925+0000","last_active":"2024-04-24T22:40:28.702124+0000","last_peered":"2024-04-24T22:40:28.702124+0000","last_clean":"2024-04-24T22:40:28.702124+0000","last_became_active":"2024-04-24T22:39:29.674681+0000","last_became_peered":"2024-04-24T22:39:29.674681+0000","last_unstale":"2024-04-24T22:40:28.702124+0000","last_undegraded":"2024-04-24T22:40:28.702124+0000","last_fullsized":"2024-04-24T22:40:28.702124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:04.295567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707157+0000","last_change":"2024-04-24T22:39:30.581061+0000","last_active":"2024-04-24T22:40:28.707157+0000","last_peered":"2024-04-24T22:40:28.707157+0000","last_clean":"2024-04-24T22:40:28.707157+0000","last_became_active":"2024-04-24T22:39:30.579807+0000","last_became_peered":"2024-04-24T22:39:30.579807+0000","last_unstale":"2024-04-24T22:40:28.707157+0000","last_undegraded":"2024-04-24T22:40:28.707157+0000","last_fullsized":"2024-04-24T22:40:28.707157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:18.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.151089+0000","last_change":"2024-04-24T22:39:29.659948+0000","last_active":"2024-04-24T22:40:29.151089+0000","last_peered":"2024-04-24T22:40:29.151089+0000","last_clean":"2024-04-24T22:40:29.151089+0000","last_became_active":"2024-04-24T22:39:29.659606+0000","last_became_peered":"2024-04-24T22:39:29.659606+0000","last_unstale":"2024-04-24T22:40:29.151089+0000","last_undegraded":"2024-04-24T22:40:29.151089+0000","last_fullsized":"2024-04-24T22:40:29.151089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:32.877670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.714880+0000","last_change":"2024-04-24T22:39:30.587740+0000","last_active":"2024-04-24T22:40:28.714880+0000","last_peered":"2024-04-24T22:40:28.714880+0000","last_clean":"2024-04-24T22:40:28.714880+0000","last_became_active":"2024-04-24T22:39:30.587063+0000","last_became_peered":"2024-04-24T22:39:30.587063+0000","last_unstale":"2024-04-24T22:40:28.714880+0000","last_undegraded":"2024-04-24T22:40:28.714880+0000","last_fullsized":"2024-04-24T22:40:28.714880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:02.704753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.769124+0000","last_change":"2024-04-24T22:39:29.672345+0000","last_active":"2024-04-24T22:40:28.769124+0000","last_peered":"2024-04-24T22:40:28.769124+0000","last_clean":"2024-04-24T22:40:28.769124+0000","last_became_active":"2024-04-24T22:39:29.671957+0000","last_became_peered":"2024-04-24T22:39:29.671957+0000","last_unstale":"2024-04-24T22:40:28.769124+0000","last_undegraded":"2024-04-24T22:40:28.769124+0000","last_fullsized":"2024-04-24T22:40:28.769124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.181431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706495+0000","last_change":"2024-04-24T22:39:30.595273+0000","last_active":"2024-04-24T22:40:28.706495+0000","last_peered":"2024-04-24T22:40:28.706495+0000","last_clean":"2024-04-24T22:40:28.706495+0000","last_became_active":"2024-04-24T22:39:30.594765+0000","last_became_peered":"2024-04-24T22:39:30.594765+0000","last_unstale":"2024-04-24T22:40:28.706495+0000","last_undegraded":"2024-04-24T22:40:28.706495+0000","last_fullsized":"2024-04-24T22:40:28.706495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.836241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923654+0000","last_change":"2024-04-24T22:39:29.663779+0000","last_active":"2024-04-24T22:40:28.923654+0000","last_peered":"2024-04-24T22:40:28.923654+0000","last_clean":"2024-04-24T22:40:28.923654+0000","last_became_active":"2024-04-24T22:39:29.663484+0000","last_became_peered":"2024-04-24T22:39:29.663484+0000","last_unstale":"2024-04-24T22:40:28.923654+0000","last_undegraded":"2024-04-24T22:40:28.923654+0000","last_fullsized":"2024-04-24T22:40:28.923654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:37.322047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701781+0000","last_change":"2024-04-24T22:39:30.568608+0000","last_active":"2024-04-24T22:40:28.701781+0000","last_peered":"2024-04-24T22:40:28.701781+0000","last_clean":"2024-04-24T22:40:28.701781+0000","last_became_active":"2024-04-24T22:39:30.568271+0000","last_became_peered":"2024-04-24T22:39:30.568271+0000","last_unstale":"2024-04-24T22:40:28.701781+0000","last_undegraded":"2024-04-24T22:40:28.701781+0000","last_fullsized":"2024-04-24T22:40:28.701781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:31.433923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.712697+0000","last_change":"2024-04-24T22:39:29.663745+0000","last_active":"2024-04-24T22:40:28.712697+0000","last_peered":"2024-04-24T22:40:28.712697+0000","last_clean":"2024-04-24T22:40:28.712697+0000","last_became_active":"2024-04-24T22:39:29.663436+0000","last_became_peered":"2024-04-24T22:39:29.663436+0000","last_unstale":"2024-04-24T22:40:28.712697+0000","last_undegraded":"2024-04-24T22:40:28.712697+0000","last_fullsized":"2024-04-24T22:40:28.712697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.502177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.703950+0000","last_change":"2024-04-24T22:39:30.580645+0000","last_active":"2024-04-24T22:40:28.703950+0000","last_peered":"2024-04-24T22:40:28.703950+0000","last_clean":"2024-04-24T22:40:28.703950+0000","last_became_active":"2024-04-24T22:39:30.580288+0000","last_became_peered":"2024-04-24T22:39:30.580288+0000","last_unstale":"2024-04-24T22:40:28.703950+0000","last_undegraded":"2024-04-24T22:40:28.703950+0000","last_fullsized":"2024-04-24T22:40:28.703950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:14.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705162+0000","last_change":"2024-04-24T22:39:29.657571+0000","last_active":"2024-04-24T22:40:28.705162+0000","last_peered":"2024-04-24T22:40:28.705162+0000","last_clean":"2024-04-24T22:40:28.705162+0000","last_became_active":"2024-04-24T22:39:29.657230+0000","last_became_peered":"2024-04-24T22:39:29.657230+0000","last_unstale":"2024-04-24T22:40:28.705162+0000","last_undegraded":"2024-04-24T22:40:28.705162+0000","last_fullsized":"2024-04-24T22:40:28.705162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:26.638528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.711186+0000","last_change":"2024-04-24T22:39:30.590468+0000","last_active":"2024-04-24T22:40:28.711186+0000","last_peered":"2024-04-24T22:40:28.711186+0000","last_clean":"2024-04-24T22:40:28.711186+0000","last_became_active":"2024-04-24T22:39:30.590157+0000","last_became_peered":"2024-04-24T22:39:30.590157+0000","last_unstale":"2024-04-24T22:40:28.711186+0000","last_undegraded":"2024-04-24T22:40:28.711186+0000","last_fullsized":"2024-04-24T22:40:28.711186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.712381+0000","last_change":"2024-04-24T22:40:28.711927+0000","last_active":"2024-04-24T22:40:28.712381+0000","last_peered":"2024-04-24T22:40:28.712381+0000","last_clean":"2024-04-24T22:40:28.712381+0000","last_became_active":"2024-04-24T22:40:28.707475+0000","last_became_peered":"2024-04-24T22:40:28.707475+0000","last_unstale":"2024-04-24T22:40:28.712381+0000","last_undegraded":"2024-04-24T22:40:28.712381+0000","last_fullsized":"2024-04-24T22:40:28.712381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:07.415691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702037+0000","last_change":"2024-04-24T22:39:30.577213+0000","last_active":"2024-04-24T22:40:28.702037+0000","last_peered":"2024-04-24T22:40:28.702037+0000","last_clean":"2024-04-24T22:40:28.702037+0000","last_became_active":"2024-04-24T22:39:30.576883+0000","last_became_peered":"2024-04-24T22:39:30.576883+0000","last_unstale":"2024-04-24T22:40:28.702037+0000","last_undegraded":"2024-04-24T22:40:28.702037+0000","last_fullsized":"2024-04-24T22:40:28.702037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:16.984591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706753+0000","last_change":"2024-04-24T22:39:29.667082+0000","last_active":"2024-04-24T22:40:28.706753+0000","last_peered":"2024-04-24T22:40:28.706753+0000","last_clean":"2024-04-24T22:40:28.706753+0000","last_became_active":"2024-04-24T22:39:29.666631+0000","last_became_peered":"2024-04-24T22:39:29.666631+0000","last_unstale":"2024-04-24T22:40:28.706753+0000","last_undegraded":"2024-04-24T22:40:28.706753+0000","last_fullsized":"2024-04-24T22:40:28.706753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.001201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702297+0000","last_change":"2024-04-24T22:39:30.565848+0000","last_active":"2024-04-24T22:40:28.702297+0000","last_peered":"2024-04-24T22:40:28.702297+0000","last_clean":"2024-04-24T22:40:28.702297+0000","last_became_active":"2024-04-24T22:39:30.565533+0000","last_became_peered":"2024-04-24T22:39:30.565533+0000","last_unstale":"2024-04-24T22:40:28.702297+0000","last_undegraded":"2024-04-24T22:40:28.702297+0000","last_fullsized":"2024-04-24T22:40:28.702297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:02.406509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078078+0000","last_change":"2024-04-24T22:39:29.662575+0000","last_active":"2024-04-24T22:40:29.078078+0000","last_peered":"2024-04-24T22:40:29.078078+0000","last_clean":"2024-04-24T22:40:29.078078+0000","last_became_active":"2024-04-24T22:39:29.662273+0000","last_became_peered":"2024-04-24T22:39:29.662273+0000","last_unstale":"2024-04-24T22:40:29.078078+0000","last_undegraded":"2024-04-24T22:40:29.078078+0000","last_fullsized":"2024-04-24T22:40:29.078078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:42.661149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701861+0000","last_change":"2024-04-24T22:39:30.587813+0000","last_active":"2024-04-24T22:40:28.701861+0000","last_peered":"2024-04-24T22:40:28.701861+0000","last_clean":"2024-04-24T22:40:28.701861+0000","last_became_active":"2024-04-24T22:39:30.587533+0000","last_became_peered":"2024-04-24T22:39:30.587533+0000","last_unstale":"2024-04-24T22:40:28.701861+0000","last_undegraded":"2024-04-24T22:40:28.701861+0000","last_fullsized":"2024-04-24T22:40:28.701861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:27.042695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707273+0000","last_change":"2024-04-24T22:39:29.664843+0000","last_active":"2024-04-24T22:40:28.707273+0000","last_peered":"2024-04-24T22:40:28.707273+0000","last_clean":"2024-04-24T22:40:28.707273+0000","last_became_active":"2024-04-24T22:39:29.664542+0000","last_became_peered":"2024-04-24T22:39:29.664542+0000","last_unstale":"2024-04-24T22:40:28.707273+0000","last_undegraded":"2024-04-24T22:40:28.707273+0000","last_fullsized":"2024-04-24T22:40:28.707273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:17.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702207+0000","last_change":"2024-04-24T22:39:30.569753+0000","last_active":"2024-04-24T22:40:28.702207+0000","last_peered":"2024-04-24T22:40:28.702207+0000","last_clean":"2024-04-24T22:40:28.702207+0000","last_became_active":"2024-04-24T22:39:30.569433+0000","last_became_peered":"2024-04-24T22:39:30.569433+0000","last_unstale":"2024-04-24T22:40:28.702207+0000","last_undegraded":"2024-04-24T22:40:28.702207+0000","last_fullsized":"2024-04-24T22:40:28.702207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704027+0000","last_change":"2024-04-24T22:39:30.574076+0000","last_active":"2024-04-24T22:40:28.704027+0000","last_peered":"2024-04-24T22:40:28.704027+0000","last_clean":"2024-04-24T22:40:28.704027+0000","last_became_active":"2024-04-24T22:39:30.573769+0000","last_became_peered":"2024-04-24T22:39:30.573769+0000","last_unstale":"2024-04-24T22:40:28.704027+0000","last_undegraded":"2024-04-24T22:40:28.704027+0000","last_fullsized":"2024-04-24T22:40:28.704027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:55.456402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705576+0000","last_change":"2024-04-24T22:39:30.570762+0000","last_active":"2024-04-24T22:40:28.705576+0000","last_peered":"2024-04-24T22:40:28.705576+0000","last_clean":"2024-04-24T22:40:28.705576+0000","last_became_active":"2024-04-24T22:39:30.570459+0000","last_became_peered":"2024-04-24T22:39:30.570459+0000","last_unstale":"2024-04-24T22:40:28.705576+0000","last_undegraded":"2024-04-24T22:40:28.705576+0000","last_fullsized":"2024-04-24T22:40:28.705576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.826752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705035+0000","last_change":"2024-04-24T22:39:30.591414+0000","last_active":"2024-04-24T22:40:28.705035+0000","last_peered":"2024-04-24T22:40:28.705035+0000","last_clean":"2024-04-24T22:40:28.705035+0000","last_became_active":"2024-04-24T22:39:30.591135+0000","last_became_peered":"2024-04-24T22:39:30.591135+0000","last_unstale":"2024-04-24T22:40:28.705035+0000","last_undegraded":"2024-04-24T22:40:28.705035+0000","last_fullsized":"2024-04-24T22:40:28.705035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:58.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.715882+0000","last_change":"2024-04-24T22:40:28.715882+0000","last_active":"2024-04-24T22:40:28.715882+0000","last_peered":"2024-04-24T22:40:28.715882+0000","last_clean":"2024-04-24T22:40:28.715882+0000","last_became_active":"2024-04-24T22:40:28.714856+0000","last_became_peered":"2024-04-24T22:40:28.714856+0000","last_unstale":"2024-04-24T22:40:28.715882+0000","last_undegraded":"2024-04-24T22:40:28.715882+0000","last_fullsized":"2024-04-24T22:40:28.715882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.841407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706825+0000","last_change":"2024-04-24T22:39:30.583955+0000","last_active":"2024-04-24T22:40:28.706825+0000","last_peered":"2024-04-24T22:40:28.706825+0000","last_clean":"2024-04-24T22:40:28.706825+0000","last_became_active":"2024-04-24T22:39:30.583632+0000","last_became_peered":"2024-04-24T22:39:30.583632+0000","last_unstale":"2024-04-24T22:40:28.706825+0000","last_undegraded":"2024-04-24T22:40:28.706825+0000","last_fullsized":"2024-04-24T22:40:28.706825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:44.150140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704151+0000","last_change":"2024-04-24T22:39:30.569989+0000","last_active":"2024-04-24T22:40:28.704151+0000","last_peered":"2024-04-24T22:40:28.704151+0000","last_clean":"2024-04-24T22:40:28.704151+0000","last_became_active":"2024-04-24T22:39:30.569680+0000","last_became_peered":"2024-04-24T22:39:30.569680+0000","last_unstale":"2024-04-24T22:40:28.704151+0000","last_undegraded":"2024-04-24T22:40:28.704151+0000","last_fullsized":"2024-04-24T22:40:28.704151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:55.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706748+0000","last_change":"2024-04-24T22:39:30.572776+0000","last_active":"2024-04-24T22:40:28.706748+0000","last_peered":"2024-04-24T22:40:28.706748+0000","last_clean":"2024-04-24T22:40:28.706748+0000","last_became_active":"2024-04-24T22:39:30.572445+0000","last_became_peered":"2024-04-24T22:39:30.572445+0000","last_unstale":"2024-04-24T22:40:28.706748+0000","last_undegraded":"2024-04-24T22:40:28.706748+0000","last_fullsized":"2024-04-24T22:40:28.706748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:40.058377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.714762+0000","last_change":"2024-04-24T22:39:30.587453+0000","last_active":"2024-04-24T22:40:28.714762+0000","last_peered":"2024-04-24T22:40:28.714762+0000","last_clean":"2024-04-24T22:40:28.714762+0000","last_became_active":"2024-04-24T22:39:30.586612+0000","last_became_peered":"2024-04-24T22:39:30.586612+0000","last_unstale":"2024-04-24T22:40:28.714762+0000","last_undegraded":"2024-04-24T22:40:28.714762+0000","last_fullsized":"2024-04-24T22:40:28.714762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:24.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077910+0000","last_change":"2024-04-24T22:39:30.592921+0000","last_active":"2024-04-24T22:40:29.077910+0000","last_peered":"2024-04-24T22:40:29.077910+0000","last_clean":"2024-04-24T22:40:29.077910+0000","last_became_active":"2024-04-24T22:39:30.592607+0000","last_became_peered":"2024-04-24T22:39:30.592607+0000","last_unstale":"2024-04-24T22:40:29.077910+0000","last_undegraded":"2024-04-24T22:40:29.077910+0000","last_fullsized":"2024-04-24T22:40:29.077910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:11.240743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.712401+0000","last_change":"2024-04-24T22:39:30.586688+0000","last_active":"2024-04-24T22:40:28.712401+0000","last_peered":"2024-04-24T22:40:28.712401+0000","last_clean":"2024-04-24T22:40:28.712401+0000","last_became_active":"2024-04-24T22:39:30.586259+0000","last_became_peered":"2024-04-24T22:39:30.586259+0000","last_unstale":"2024-04-24T22:40:28.712401+0000","last_undegraded":"2024-04-24T22:40:28.712401+0000","last_fullsized":"2024-04-24T22:40:28.712401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:24.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923812+0000","last_change":"2024-04-24T22:39:30.591940+0000","last_active":"2024-04-24T22:40:28.923812+0000","last_peered":"2024-04-24T22:40:28.923812+0000","last_clean":"2024-04-24T22:40:28.923812+0000","last_became_active":"2024-04-24T22:39:30.591635+0000","last_became_peered":"2024-04-24T22:39:30.591635+0000","last_unstale":"2024-04-24T22:40:28.923812+0000","last_undegraded":"2024-04-24T22:40:28.923812+0000","last_fullsized":"2024-04-24T22:40:28.923812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:16.097707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.711166+0000","last_change":"2024-04-24T22:39:30.588188+0000","last_active":"2024-04-24T22:40:28.711166+0000","last_peered":"2024-04-24T22:40:28.711166+0000","last_clean":"2024-04-24T22:40:28.711166+0000","last_became_active":"2024-04-24T22:39:30.587608+0000","last_became_peered":"2024-04-24T22:39:30.587608+0000","last_unstale":"2024-04-24T22:40:28.711166+0000","last_undegraded":"2024-04-24T22:40:28.711166+0000","last_fullsized":"2024-04-24T22:40:28.711166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:17.690009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704225+0000","last_change":"2024-04-24T22:39:30.567467+0000","last_active":"2024-04-24T22:40:28.704225+0000","last_peered":"2024-04-24T22:40:28.704225+0000","last_clean":"2024-04-24T22:40:28.704225+0000","last_became_active":"2024-04-24T22:39:30.567162+0000","last_became_peered":"2024-04-24T22:39:30.567162+0000","last_unstale":"2024-04-24T22:40:28.704225+0000","last_undegraded":"2024-04-24T22:40:28.704225+0000","last_fullsized":"2024-04-24T22:40:28.704225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:25.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63466,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":502,"num_read_kb":317,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":456231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1107534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1552384,"data_stored":1048283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":515080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":516284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1564672,"data_stored":1047576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:41:00.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T22:41:00.059 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:59 smithi171 ceph-mon[32337]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T22:41:00.059 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:00.059 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:00.059 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:40:59 smithi171 ceph-mon[32337]: from='mgr.14152 172.21.15.111:0/1984050521' entity='mgr.x' 2024-04-24T22:41:00.539 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:00.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:00 smithi186 ceph-mon[32753]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:00 smithi186 ceph-mon[32753]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T22:41:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:00 smithi111 ceph-mon[27746]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:01.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:00 smithi111 ceph-mon[27746]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T22:41:01.075 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:00 smithi171 ceph-mon[32337]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:01.075 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:00 smithi171 ceph-mon[32337]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 -- 172.21.15.111:0/1679511686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 msgr2=0x7fbdb4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 --2- 172.21.15.111:0/1679511686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb4106dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0099c0 tx=0x7fbd9c009cd0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 -- 172.21.15.111:0/1679511686 shutdown_connections 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 --2- 172.21.15.111:0/1679511686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41074a0 0x7fbdb410f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 --2- 172.21.15.111:0/1679511686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41023c0 0x7fbdb41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 --2- 172.21.15.111:0/1679511686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb4106dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.798+0000 7fbdb9200700 1 -- 172.21.15.111:0/1679511686 >> 172.21.15.111:0/1679511686 conn(0x7fbdb40fdcb0 msgr2=0x7fbdb41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:02.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.799+0000 7fbdb9200700 1 -- 172.21.15.111:0/1679511686 shutdown_connections 2024-04-24T22:41:02.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.799+0000 7fbdb9200700 1 -- 172.21.15.111:0/1679511686 wait complete. 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.800+0000 7fbdb9200700 1 Processor -- start 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.800+0000 7fbdb9200700 1 -- start start 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.800+0000 7fbdb9200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.800+0000 7fbdb9200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb9200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 0x7fbdb419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:02.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb9200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41124b0 con 0x7fbdb4102d70 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb9200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb4112330 con 0x7fbdb41074a0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb9200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb4112630 con 0x7fbdb41023c0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb2a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34996/0 (socket says 172.21.15.111:34996) 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 -- 172.21.15.111:0/2928174508 learned_addr learned my addr 172.21.15.111:0/2928174508 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3e00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 0x7fbdb419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 -- 172.21.15.111:0/2928174508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 msgr2=0x7fbdb419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 0x7fbdb419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 -- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 msgr2=0x7fbdb41a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb41a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3400700 1 -- 172.21.15.111:0/2928174508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb419b160 con 0x7fbdb41023c0 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb3e00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 0x7fbdb419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.801+0000 7fbdb2a00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb41a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:02.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdb3400700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9a0 tx=0x7fbda400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda400cbd0 con 0x7fbdb41023c0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda400cd70 con 0x7fbdb41023c0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb419b450 con 0x7fbdb41023c0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4018750 con 0x7fbdb41023c0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.802+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41a7710 con 0x7fbdb41023c0 2024-04-24T22:41:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.803+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4068f20 con 0x7fbdb41023c0 2024-04-24T22:41:02.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.805+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbda4010430 con 0x7fbdb41023c0 2024-04-24T22:41:02.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.806+0000 7fbdabe00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 0x7fbd980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:02.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.807+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbda40cf2f0 con 0x7fbdb41023c0 2024-04-24T22:41:02.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.807+0000 7fbdb2a00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 0x7fbd980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:02.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.807+0000 7fbdb2a00700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 0x7fbd980a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:02.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:02.811+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbda409c200 con 0x7fbdb41023c0 2024-04-24T22:41:02.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:02 smithi186 ceph-mon[32753]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:03.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.031+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdb4075470 con 0x7fbd980a2ca0 2024-04-24T22:41:03.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:02 smithi111 ceph-mon[27746]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:03.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.039+0000 7fbdabe00700 1 -- 172.21.15.111:0/2928174508 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258917 (secure 0 0 0) 0x7fbdb4075470 con 0x7fbd980a2ca0 2024-04-24T22:41:03.042 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:03.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 msgr2=0x7fbd980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 0x7fbd980a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c005fd0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 msgr2=0x7fbdb41a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9a0 tx=0x7fbda400ecb0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 shutdown_connections 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbd980a2ca0 0x7fbd980a5160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb41074a0 0x7fbdb419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb41023c0 0x7fbdb41a0840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 --2- 172.21.15.111:0/2928174508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb4102d70 0x7fbdb41a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.046+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 >> 172.21.15.111:0/2928174508 conn(0x7fbdb40fdcb0 msgr2=0x7fbdb410ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.047+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 shutdown_connections 2024-04-24T22:41:03.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:03.047+0000 7fbdb9200700 1 -- 172.21.15.111:0/2928174508 wait complete. 2024-04-24T22:41:03.051 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-04-24T22:41:03.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:02 smithi171 ceph-mon[32337]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:03.932 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-24T22:41:01.604339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654458,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":664,"num_read_kb":452,"num_write":347,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":625,"ondisk_log_size":625,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383148,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558676,"statfs":{"total":1151940427776,"available":1151548084224,"internally_reserved":0,"allocated":13479936,"data_stored":7430472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":41,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704358+0000","last_change":"2024-04-24T22:39:30.580615+0000","last_active":"2024-04-24T22:40:28.704358+0000","last_peered":"2024-04-24T22:40:28.704358+0000","last_clean":"2024-04-24T22:40:28.704358+0000","last_became_active":"2024-04-24T22:39:30.580299+0000","last_became_peered":"2024-04-24T22:39:30.580299+0000","last_unstale":"2024-04-24T22:40:28.704358+0000","last_undegraded":"2024-04-24T22:40:28.704358+0000","last_fullsized":"2024-04-24T22:40:28.704358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.364115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.579302+0000","last_change":"2024-04-24T22:39:30.588495+0000","last_active":"2024-04-24T22:40:57.579302+0000","last_peered":"2024-04-24T22:40:57.579302+0000","last_clean":"2024-04-24T22:40:57.579302+0000","last_became_active":"2024-04-24T22:39:30.585588+0000","last_became_peered":"2024-04-24T22:39:30.585588+0000","last_unstale":"2024-04-24T22:40:57.579302+0000","last_undegraded":"2024-04-24T22:40:57.579302+0000","last_fullsized":"2024-04-24T22:40:57.579302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.894832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.592649+0000","last_change":"2024-04-24T22:39:30.585139+0000","last_active":"2024-04-24T22:40:57.592649+0000","last_peered":"2024-04-24T22:40:57.592649+0000","last_clean":"2024-04-24T22:40:57.592649+0000","last_became_active":"2024-04-24T22:39:30.583299+0000","last_became_peered":"2024-04-24T22:39:30.583299+0000","last_unstale":"2024-04-24T22:40:57.592649+0000","last_undegraded":"2024-04-24T22:40:57.592649+0000","last_fullsized":"2024-04-24T22:40:57.592649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:20.941383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.561596+0000","last_change":"2024-04-24T22:39:30.566540+0000","last_active":"2024-04-24T22:40:57.561596+0000","last_peered":"2024-04-24T22:40:57.561596+0000","last_clean":"2024-04-24T22:40:57.561596+0000","last_became_active":"2024-04-24T22:39:30.566195+0000","last_became_peered":"2024-04-24T22:39:30.566195+0000","last_unstale":"2024-04-24T22:40:57.561596+0000","last_undegraded":"2024-04-24T22:40:57.561596+0000","last_fullsized":"2024-04-24T22:40:57.561596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:24.538153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.587733+0000","last_change":"2024-04-24T22:39:30.591842+0000","last_active":"2024-04-24T22:40:57.587733+0000","last_peered":"2024-04-24T22:40:57.587733+0000","last_clean":"2024-04-24T22:40:57.587733+0000","last_became_active":"2024-04-24T22:39:30.591579+0000","last_became_peered":"2024-04-24T22:39:30.591579+0000","last_unstale":"2024-04-24T22:40:57.587733+0000","last_undegraded":"2024-04-24T22:40:57.587733+0000","last_fullsized":"2024-04-24T22:40:57.587733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:48.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.567004+0000","last_change":"2024-04-24T22:39:30.577689+0000","last_active":"2024-04-24T22:40:57.567004+0000","last_peered":"2024-04-24T22:40:57.567004+0000","last_clean":"2024-04-24T22:40:57.567004+0000","last_became_active":"2024-04-24T22:39:30.577362+0000","last_became_peered":"2024-04-24T22:39:30.577362+0000","last_unstale":"2024-04-24T22:40:57.567004+0000","last_undegraded":"2024-04-24T22:40:57.567004+0000","last_fullsized":"2024-04-24T22:40:57.567004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:52.808898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701687+0000","last_change":"2024-04-24T22:39:30.576400+0000","last_active":"2024-04-24T22:40:28.701687+0000","last_peered":"2024-04-24T22:40:28.701687+0000","last_clean":"2024-04-24T22:40:28.701687+0000","last_became_active":"2024-04-24T22:39:30.576141+0000","last_became_peered":"2024-04-24T22:39:30.576141+0000","last_unstale":"2024-04-24T22:40:28.701687+0000","last_undegraded":"2024-04-24T22:40:28.701687+0000","last_fullsized":"2024-04-24T22:40:28.701687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.582284+0000","last_change":"2024-04-24T22:39:30.594496+0000","last_active":"2024-04-24T22:40:57.582284+0000","last_peered":"2024-04-24T22:40:57.582284+0000","last_clean":"2024-04-24T22:40:57.582284+0000","last_became_active":"2024-04-24T22:39:30.589541+0000","last_became_peered":"2024-04-24T22:39:30.589541+0000","last_unstale":"2024-04-24T22:40:57.582284+0000","last_undegraded":"2024-04-24T22:40:57.582284+0000","last_fullsized":"2024-04-24T22:40:57.582284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:50.593142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.573732+0000","last_change":"2024-04-24T22:39:30.567922+0000","last_active":"2024-04-24T22:40:57.573732+0000","last_peered":"2024-04-24T22:40:57.573732+0000","last_clean":"2024-04-24T22:40:57.573732+0000","last_became_active":"2024-04-24T22:39:30.567470+0000","last_became_peered":"2024-04-24T22:39:30.567470+0000","last_unstale":"2024-04-24T22:40:57.573732+0000","last_undegraded":"2024-04-24T22:40:57.573732+0000","last_fullsized":"2024-04-24T22:40:57.573732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:11.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.571977+0000","last_change":"2024-04-24T22:40:29.152804+0000","last_active":"2024-04-24T22:40:57.571977+0000","last_peered":"2024-04-24T22:40:57.571977+0000","last_clean":"2024-04-24T22:40:57.571977+0000","last_became_active":"2024-04-24T22:40:29.152234+0000","last_became_peered":"2024-04-24T22:40:29.152234+0000","last_unstale":"2024-04-24T22:40:57.571977+0000","last_undegraded":"2024-04-24T22:40:57.571977+0000","last_fullsized":"2024-04-24T22:40:57.571977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:39.167469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702237+0000","last_change":"2024-04-24T22:39:30.571679+0000","last_active":"2024-04-24T22:40:28.702237+0000","last_peered":"2024-04-24T22:40:28.702237+0000","last_clean":"2024-04-24T22:40:28.702237+0000","last_became_active":"2024-04-24T22:39:30.571375+0000","last_became_peered":"2024-04-24T22:39:30.571375+0000","last_unstale":"2024-04-24T22:40:28.702237+0000","last_undegraded":"2024-04-24T22:40:28.702237+0000","last_fullsized":"2024-04-24T22:40:28.702237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:47.219082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.558408+0000","last_change":"2024-04-24T22:39:30.565158+0000","last_active":"2024-04-24T22:40:57.558408+0000","last_peered":"2024-04-24T22:40:57.558408+0000","last_clean":"2024-04-24T22:40:57.558408+0000","last_became_active":"2024-04-24T22:39:30.564808+0000","last_became_peered":"2024-04-24T22:39:30.564808+0000","last_unstale":"2024-04-24T22:40:57.558408+0000","last_undegraded":"2024-04-24T22:40:57.558408+0000","last_fullsized":"2024-04-24T22:40:57.558408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:33.534801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.596044+0000","last_change":"2024-04-24T22:39:30.561125+0000","last_active":"2024-04-24T22:40:57.596044+0000","last_peered":"2024-04-24T22:40:57.596044+0000","last_clean":"2024-04-24T22:40:57.596044+0000","last_became_active":"2024-04-24T22:39:30.559473+0000","last_became_peered":"2024-04-24T22:39:30.559473+0000","last_unstale":"2024-04-24T22:40:57.596044+0000","last_undegraded":"2024-04-24T22:40:57.596044+0000","last_fullsized":"2024-04-24T22:40:57.596044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:09.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.592907+0000","last_change":"2024-04-24T22:39:30.582371+0000","last_active":"2024-04-24T22:40:57.592907+0000","last_peered":"2024-04-24T22:40:57.592907+0000","last_clean":"2024-04-24T22:40:57.592907+0000","last_became_active":"2024-04-24T22:39:30.582002+0000","last_became_peered":"2024-04-24T22:39:30.582002+0000","last_unstale":"2024-04-24T22:40:57.592907+0000","last_undegraded":"2024-04-24T22:40:57.592907+0000","last_fullsized":"2024-04-24T22:40:57.592907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:11.812461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.573797+0000","last_change":"2024-04-24T22:40:28.716114+0000","last_active":"2024-04-24T22:40:57.573797+0000","last_peered":"2024-04-24T22:40:57.573797+0000","last_clean":"2024-04-24T22:40:57.573797+0000","last_became_active":"2024-04-24T22:40:28.715344+0000","last_became_peered":"2024-04-24T22:40:28.715344+0000","last_unstale":"2024-04-24T22:40:57.573797+0000","last_undegraded":"2024-04-24T22:40:57.573797+0000","last_fullsized":"2024-04-24T22:40:57.573797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:31.943912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.591876+0000","last_change":"2024-04-24T22:39:30.578310+0000","last_active":"2024-04-24T22:40:57.591876+0000","last_peered":"2024-04-24T22:40:57.591876+0000","last_clean":"2024-04-24T22:40:57.591876+0000","last_became_active":"2024-04-24T22:39:30.577917+0000","last_became_peered":"2024-04-24T22:39:30.577917+0000","last_unstale":"2024-04-24T22:40:57.591876+0000","last_undegraded":"2024-04-24T22:40:57.591876+0000","last_fullsized":"2024-04-24T22:40:57.591876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:51.766196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705817+0000","last_change":"2024-04-24T22:39:30.580939+0000","last_active":"2024-04-24T22:40:28.705817+0000","last_peered":"2024-04-24T22:40:28.705817+0000","last_clean":"2024-04-24T22:40:28.705817+0000","last_became_active":"2024-04-24T22:39:30.580583+0000","last_became_peered":"2024-04-24T22:39:30.580583+0000","last_unstale":"2024-04-24T22:40:28.705817+0000","last_undegraded":"2024-04-24T22:40:28.705817+0000","last_fullsized":"2024-04-24T22:40:28.705817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.891103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.576748+0000","last_change":"2024-04-24T22:39:30.577933+0000","last_active":"2024-04-24T22:40:57.576748+0000","last_peered":"2024-04-24T22:40:57.576748+0000","last_clean":"2024-04-24T22:40:57.576748+0000","last_became_active":"2024-04-24T22:39:30.577545+0000","last_became_peered":"2024-04-24T22:39:30.577545+0000","last_unstale":"2024-04-24T22:40:57.576748+0000","last_undegraded":"2024-04-24T22:40:57.576748+0000","last_fullsized":"2024-04-24T22:40:57.576748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:26.876281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.581022+0000","last_change":"2024-04-24T22:39:30.586170+0000","last_active":"2024-04-24T22:40:57.581022+0000","last_peered":"2024-04-24T22:40:57.581022+0000","last_clean":"2024-04-24T22:40:57.581022+0000","last_became_active":"2024-04-24T22:39:30.585922+0000","last_became_peered":"2024-04-24T22:39:30.585922+0000","last_unstale":"2024-04-24T22:40:57.581022+0000","last_undegraded":"2024-04-24T22:40:57.581022+0000","last_fullsized":"2024-04-24T22:40:57.581022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:53.166725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.150655+0000","last_change":"2024-04-24T22:39:29.660103+0000","last_active":"2024-04-24T22:40:29.150655+0000","last_peered":"2024-04-24T22:40:29.150655+0000","last_clean":"2024-04-24T22:40:29.150655+0000","last_became_active":"2024-04-24T22:39:29.659809+0000","last_became_peered":"2024-04-24T22:39:29.659809+0000","last_unstale":"2024-04-24T22:40:29.150655+0000","last_undegraded":"2024-04-24T22:40:29.150655+0000","last_fullsized":"2024-04-24T22:40:29.150655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:40.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.577864+0000","last_change":"2024-04-24T22:39:30.587041+0000","last_active":"2024-04-24T22:40:57.577864+0000","last_peered":"2024-04-24T22:40:57.577864+0000","last_clean":"2024-04-24T22:40:57.577864+0000","last_became_active":"2024-04-24T22:39:30.586568+0000","last_became_peered":"2024-04-24T22:39:30.586568+0000","last_unstale":"2024-04-24T22:40:57.577864+0000","last_undegraded":"2024-04-24T22:40:57.577864+0000","last_fullsized":"2024-04-24T22:40:57.577864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:13.305790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.715537+0000","last_change":"2024-04-24T22:39:29.659591+0000","last_active":"2024-04-24T22:40:28.715537+0000","last_peered":"2024-04-24T22:40:28.715537+0000","last_clean":"2024-04-24T22:40:28.715537+0000","last_became_active":"2024-04-24T22:39:29.659281+0000","last_became_peered":"2024-04-24T22:39:29.659281+0000","last_unstale":"2024-04-24T22:40:28.715537+0000","last_undegraded":"2024-04-24T22:40:28.715537+0000","last_fullsized":"2024-04-24T22:40:28.715537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:07.909300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.594144+0000","last_change":"2024-04-24T22:39:30.589041+0000","last_active":"2024-04-24T22:40:57.594144+0000","last_peered":"2024-04-24T22:40:57.594144+0000","last_clean":"2024-04-24T22:40:57.594144+0000","last_became_active":"2024-04-24T22:39:30.588706+0000","last_became_peered":"2024-04-24T22:39:30.588706+0000","last_unstale":"2024-04-24T22:40:57.594144+0000","last_undegraded":"2024-04-24T22:40:57.594144+0000","last_fullsized":"2024-04-24T22:40:57.594144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:19.594236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702327+0000","last_change":"2024-04-24T22:39:29.659280+0000","last_active":"2024-04-24T22:40:28.702327+0000","last_peered":"2024-04-24T22:40:28.702327+0000","last_clean":"2024-04-24T22:40:28.702327+0000","last_became_active":"2024-04-24T22:39:29.658922+0000","last_became_peered":"2024-04-24T22:39:29.658922+0000","last_unstale":"2024-04-24T22:40:28.702327+0000","last_undegraded":"2024-04-24T22:40:28.702327+0000","last_fullsized":"2024-04-24T22:40:28.702327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:58.157675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.590509+0000","last_change":"2024-04-24T22:39:30.573769+0000","last_active":"2024-04-24T22:40:57.590509+0000","last_peered":"2024-04-24T22:40:57.590509+0000","last_clean":"2024-04-24T22:40:57.590509+0000","last_became_active":"2024-04-24T22:39:30.573405+0000","last_became_peered":"2024-04-24T22:39:30.573405+0000","last_unstale":"2024-04-24T22:40:57.590509+0000","last_undegraded":"2024-04-24T22:40:57.590509+0000","last_fullsized":"2024-04-24T22:40:57.590509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:08.919808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702136+0000","last_change":"2024-04-24T22:39:29.653759+0000","last_active":"2024-04-24T22:40:28.702136+0000","last_peered":"2024-04-24T22:40:28.702136+0000","last_clean":"2024-04-24T22:40:28.702136+0000","last_became_active":"2024-04-24T22:39:29.653060+0000","last_became_peered":"2024-04-24T22:39:29.653060+0000","last_unstale":"2024-04-24T22:40:28.702136+0000","last_undegraded":"2024-04-24T22:40:28.702136+0000","last_fullsized":"2024-04-24T22:40:28.702136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:02.503014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078237+0000","last_change":"2024-04-24T22:39:30.563070+0000","last_active":"2024-04-24T22:40:29.078237+0000","last_peered":"2024-04-24T22:40:29.078237+0000","last_clean":"2024-04-24T22:40:29.078237+0000","last_became_active":"2024-04-24T22:39:30.562736+0000","last_became_peered":"2024-04-24T22:39:30.562736+0000","last_unstale":"2024-04-24T22:40:29.078237+0000","last_undegraded":"2024-04-24T22:40:29.078237+0000","last_fullsized":"2024-04-24T22:40:29.078237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:15.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:55.114254+0000","last_change":"2024-04-24T22:39:29.657659+0000","last_active":"2024-04-24T22:40:55.114254+0000","last_peered":"2024-04-24T22:40:55.114254+0000","last_clean":"2024-04-24T22:40:55.114254+0000","last_became_active":"2024-04-24T22:39:29.657237+0000","last_became_peered":"2024-04-24T22:39:29.657237+0000","last_unstale":"2024-04-24T22:40:55.114254+0000","last_undegraded":"2024-04-24T22:40:55.114254+0000","last_fullsized":"2024-04-24T22:40:55.114254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:11.376959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.564801+0000","last_change":"2024-04-24T22:39:30.577542+0000","last_active":"2024-04-24T22:40:57.564801+0000","last_peered":"2024-04-24T22:40:57.564801+0000","last_clean":"2024-04-24T22:40:57.564801+0000","last_became_active":"2024-04-24T22:39:30.577194+0000","last_became_peered":"2024-04-24T22:39:30.577194+0000","last_unstale":"2024-04-24T22:40:57.564801+0000","last_undegraded":"2024-04-24T22:40:57.564801+0000","last_fullsized":"2024-04-24T22:40:57.564801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:26.000787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707141+0000","last_change":"2024-04-24T22:39:29.671709+0000","last_active":"2024-04-24T22:40:28.707141+0000","last_peered":"2024-04-24T22:40:28.707141+0000","last_clean":"2024-04-24T22:40:28.707141+0000","last_became_active":"2024-04-24T22:39:29.671306+0000","last_became_peered":"2024-04-24T22:39:29.671306+0000","last_unstale":"2024-04-24T22:40:28.707141+0000","last_undegraded":"2024-04-24T22:40:28.707141+0000","last_fullsized":"2024-04-24T22:40:28.707141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:33.833217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078035+0000","last_change":"2024-04-24T22:39:30.583720+0000","last_active":"2024-04-24T22:40:29.078035+0000","last_peered":"2024-04-24T22:40:29.078035+0000","last_clean":"2024-04-24T22:40:29.078035+0000","last_became_active":"2024-04-24T22:39:30.583468+0000","last_became_peered":"2024-04-24T22:39:30.583468+0000","last_unstale":"2024-04-24T22:40:29.078035+0000","last_undegraded":"2024-04-24T22:40:29.078035+0000","last_fullsized":"2024-04-24T22:40:29.078035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:07.679284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.709760+0000","last_change":"2024-04-24T22:39:29.669100+0000","last_active":"2024-04-24T22:40:28.709760+0000","last_peered":"2024-04-24T22:40:28.709760+0000","last_clean":"2024-04-24T22:40:28.709760+0000","last_became_active":"2024-04-24T22:39:29.668773+0000","last_became_peered":"2024-04-24T22:39:29.668773+0000","last_unstale":"2024-04-24T22:40:28.709760+0000","last_undegraded":"2024-04-24T22:40:28.709760+0000","last_fullsized":"2024-04-24T22:40:28.709760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:56.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.585897+0000","last_change":"2024-04-24T22:39:30.594562+0000","last_active":"2024-04-24T22:40:57.585897+0000","last_peered":"2024-04-24T22:40:57.585897+0000","last_clean":"2024-04-24T22:40:57.585897+0000","last_became_active":"2024-04-24T22:39:30.589928+0000","last_became_peered":"2024-04-24T22:39:30.589928+0000","last_unstale":"2024-04-24T22:40:57.585897+0000","last_undegraded":"2024-04-24T22:40:57.585897+0000","last_fullsized":"2024-04-24T22:40:57.585897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:14.951690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077769+0000","last_change":"2024-04-24T22:39:29.658615+0000","last_active":"2024-04-24T22:40:29.077769+0000","last_peered":"2024-04-24T22:40:29.077769+0000","last_clean":"2024-04-24T22:40:29.077769+0000","last_became_active":"2024-04-24T22:39:29.658178+0000","last_became_peered":"2024-04-24T22:39:29.658178+0000","last_unstale":"2024-04-24T22:40:29.077769+0000","last_undegraded":"2024-04-24T22:40:29.077769+0000","last_fullsized":"2024-04-24T22:40:29.077769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.821509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.588189+0000","last_change":"2024-04-24T22:39:30.592142+0000","last_active":"2024-04-24T22:40:57.588189+0000","last_peered":"2024-04-24T22:40:57.588189+0000","last_clean":"2024-04-24T22:40:57.588189+0000","last_became_active":"2024-04-24T22:39:30.591417+0000","last_became_peered":"2024-04-24T22:39:30.591417+0000","last_unstale":"2024-04-24T22:40:57.588189+0000","last_undegraded":"2024-04-24T22:40:57.588189+0000","last_fullsized":"2024-04-24T22:40:57.588189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:55.818795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:41:00.120590+0000","last_change":"2024-04-24T22:39:29.662513+0000","last_active":"2024-04-24T22:41:00.120590+0000","last_peered":"2024-04-24T22:41:00.120590+0000","last_clean":"2024-04-24T22:41:00.120590+0000","last_became_active":"2024-04-24T22:39:29.662223+0000","last_became_peered":"2024-04-24T22:39:29.662223+0000","last_unstale":"2024-04-24T22:41:00.120590+0000","last_undegraded":"2024-04-24T22:41:00.120590+0000","last_fullsized":"2024-04-24T22:41:00.120590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.012595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.574220+0000","last_change":"2024-04-24T22:39:30.577594+0000","last_active":"2024-04-24T22:40:57.574220+0000","last_peered":"2024-04-24T22:40:57.574220+0000","last_clean":"2024-04-24T22:40:57.574220+0000","last_became_active":"2024-04-24T22:39:30.577254+0000","last_became_peered":"2024-04-24T22:39:30.577254+0000","last_unstale":"2024-04-24T22:40:57.574220+0000","last_undegraded":"2024-04-24T22:40:57.574220+0000","last_fullsized":"2024-04-24T22:40:57.574220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:47.926878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701897+0000","last_change":"2024-04-24T22:39:29.659898+0000","last_active":"2024-04-24T22:40:28.701897+0000","last_peered":"2024-04-24T22:40:28.701897+0000","last_clean":"2024-04-24T22:40:28.701897+0000","last_became_active":"2024-04-24T22:39:29.659582+0000","last_became_peered":"2024-04-24T22:39:29.659582+0000","last_unstale":"2024-04-24T22:40:28.701897+0000","last_undegraded":"2024-04-24T22:40:28.701897+0000","last_fullsized":"2024-04-24T22:40:28.701897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.592186+0000","last_change":"2024-04-24T22:39:30.588271+0000","last_active":"2024-04-24T22:40:57.592186+0000","last_peered":"2024-04-24T22:40:57.592186+0000","last_clean":"2024-04-24T22:40:57.592186+0000","last_became_active":"2024-04-24T22:39:30.578846+0000","last_became_peered":"2024-04-24T22:39:30.578846+0000","last_unstale":"2024-04-24T22:40:57.592186+0000","last_undegraded":"2024-04-24T22:40:57.592186+0000","last_fullsized":"2024-04-24T22:40:57.592186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:23.807550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078249+0000","last_change":"2024-04-24T22:39:29.654708+0000","last_active":"2024-04-24T22:40:29.078249+0000","last_peered":"2024-04-24T22:40:29.078249+0000","last_clean":"2024-04-24T22:40:29.078249+0000","last_became_active":"2024-04-24T22:39:29.654290+0000","last_became_peered":"2024-04-24T22:39:29.654290+0000","last_unstale":"2024-04-24T22:40:29.078249+0000","last_undegraded":"2024-04-24T22:40:29.078249+0000","last_fullsized":"2024-04-24T22:40:29.078249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:29.445700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.573037+0000","last_change":"2024-04-24T22:39:30.569436+0000","last_active":"2024-04-24T22:40:57.573037+0000","last_peered":"2024-04-24T22:40:57.573037+0000","last_clean":"2024-04-24T22:40:57.573037+0000","last_became_active":"2024-04-24T22:39:30.569104+0000","last_became_peered":"2024-04-24T22:39:30.569104+0000","last_unstale":"2024-04-24T22:40:57.573037+0000","last_undegraded":"2024-04-24T22:40:57.573037+0000","last_fullsized":"2024-04-24T22:40:57.573037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:30.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":229,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:41:01.121929+0000","last_change":"2024-04-24T22:39:29.657883+0000","last_active":"2024-04-24T22:41:01.121929+0000","last_peered":"2024-04-24T22:41:01.121929+0000","last_clean":"2024-04-24T22:41:01.121929+0000","last_became_active":"2024-04-24T22:39:29.657496+0000","last_became_peered":"2024-04-24T22:39:29.657496+0000","last_unstale":"2024-04-24T22:41:01.121929+0000","last_undegraded":"2024-04-24T22:41:01.121929+0000","last_fullsized":"2024-04-24T22:41:01.121929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:51:32.037937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59714,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":148,"num_write":31,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.572957+0000","last_change":"2024-04-24T22:39:30.569744+0000","last_active":"2024-04-24T22:40:57.572957+0000","last_peered":"2024-04-24T22:40:57.572957+0000","last_clean":"2024-04-24T22:40:57.572957+0000","last_became_active":"2024-04-24T22:39:30.569392+0000","last_became_peered":"2024-04-24T22:39:30.569392+0000","last_unstale":"2024-04-24T22:40:57.572957+0000","last_undegraded":"2024-04-24T22:40:57.572957+0000","last_fullsized":"2024-04-24T22:40:57.572957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:09.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.725538+0000","last_change":"2024-04-24T22:40:28.725141+0000","last_active":"2024-04-24T22:40:28.725538+0000","last_peered":"2024-04-24T22:40:28.725538+0000","last_clean":"2024-04-24T22:40:28.725538+0000","last_became_active":"2024-04-24T22:40:28.713510+0000","last_became_peered":"2024-04-24T22:40:28.713510+0000","last_unstale":"2024-04-24T22:40:28.725538+0000","last_undegraded":"2024-04-24T22:40:28.725538+0000","last_fullsized":"2024-04-24T22:40:28.725538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:09.796454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.585259+0000","last_change":"2024-04-24T22:40:29.153608+0000","last_active":"2024-04-24T22:40:57.585259+0000","last_peered":"2024-04-24T22:40:57.585259+0000","last_clean":"2024-04-24T22:40:57.585259+0000","last_became_active":"2024-04-24T22:40:29.152328+0000","last_became_peered":"2024-04-24T22:40:29.152328+0000","last_unstale":"2024-04-24T22:40:57.585259+0000","last_undegraded":"2024-04-24T22:40:57.585259+0000","last_fullsized":"2024-04-24T22:40:57.585259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:15.265528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077873+0000","last_change":"2024-04-24T22:39:29.657561+0000","last_active":"2024-04-24T22:40:29.077873+0000","last_peered":"2024-04-24T22:40:29.077873+0000","last_clean":"2024-04-24T22:40:29.077873+0000","last_became_active":"2024-04-24T22:39:29.657261+0000","last_became_peered":"2024-04-24T22:39:29.657261+0000","last_unstale":"2024-04-24T22:40:29.077873+0000","last_undegraded":"2024-04-24T22:40:29.077873+0000","last_fullsized":"2024-04-24T22:40:29.077873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:11.856640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701777+0000","last_change":"2024-04-24T22:39:30.577405+0000","last_active":"2024-04-24T22:40:28.701777+0000","last_peered":"2024-04-24T22:40:28.701777+0000","last_clean":"2024-04-24T22:40:28.701777+0000","last_became_active":"2024-04-24T22:39:30.576980+0000","last_became_peered":"2024-04-24T22:39:30.576980+0000","last_unstale":"2024-04-24T22:40:28.701777+0000","last_undegraded":"2024-04-24T22:40:28.701777+0000","last_fullsized":"2024-04-24T22:40:28.701777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:00.145497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.183327+0000","last_change":"2024-04-24T22:39:29.661054+0000","last_active":"2024-04-24T22:40:57.183327+0000","last_peered":"2024-04-24T22:40:57.183327+0000","last_clean":"2024-04-24T22:40:57.183327+0000","last_became_active":"2024-04-24T22:39:29.660638+0000","last_became_peered":"2024-04-24T22:39:29.660638+0000","last_unstale":"2024-04-24T22:40:57.183327+0000","last_undegraded":"2024-04-24T22:40:57.183327+0000","last_fullsized":"2024-04-24T22:40:57.183327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:51.891654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701595+0000","last_change":"2024-04-24T22:39:30.570407+0000","last_active":"2024-04-24T22:40:28.701595+0000","last_peered":"2024-04-24T22:40:28.701595+0000","last_clean":"2024-04-24T22:40:28.701595+0000","last_became_active":"2024-04-24T22:39:30.570070+0000","last_became_peered":"2024-04-24T22:39:30.570070+0000","last_unstale":"2024-04-24T22:40:28.701595+0000","last_undegraded":"2024-04-24T22:40:28.701595+0000","last_fullsized":"2024-04-24T22:40:28.701595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.785122+0000","last_change":"2024-04-24T22:40:28.785122+0000","last_active":"2024-04-24T22:40:28.785122+0000","last_peered":"2024-04-24T22:40:28.785122+0000","last_clean":"2024-04-24T22:40:28.785122+0000","last_became_active":"2024-04-24T22:40:28.770463+0000","last_became_peered":"2024-04-24T22:40:28.770463+0000","last_unstale":"2024-04-24T22:40:28.785122+0000","last_undegraded":"2024-04-24T22:40:28.785122+0000","last_fullsized":"2024-04-24T22:40:28.785122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:18.755290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077842+0000","last_change":"2024-04-24T22:39:30.562092+0000","last_active":"2024-04-24T22:40:29.077842+0000","last_peered":"2024-04-24T22:40:29.077842+0000","last_clean":"2024-04-24T22:40:29.077842+0000","last_became_active":"2024-04-24T22:39:30.561704+0000","last_became_peered":"2024-04-24T22:39:30.561704+0000","last_unstale":"2024-04-24T22:40:29.077842+0000","last_undegraded":"2024-04-24T22:40:29.077842+0000","last_fullsized":"2024-04-24T22:40:29.077842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.996837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702283+0000","last_change":"2024-04-24T22:39:29.655612+0000","last_active":"2024-04-24T22:40:28.702283+0000","last_peered":"2024-04-24T22:40:28.702283+0000","last_clean":"2024-04-24T22:40:28.702283+0000","last_became_active":"2024-04-24T22:39:29.655212+0000","last_became_peered":"2024-04-24T22:39:29.655212+0000","last_unstale":"2024-04-24T22:40:28.702283+0000","last_undegraded":"2024-04-24T22:40:28.702283+0000","last_fullsized":"2024-04-24T22:40:28.702283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.414774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.152885+0000","last_change":"2024-04-24T22:40:29.152885+0000","last_active":"2024-04-24T22:40:29.152885+0000","last_peered":"2024-04-24T22:40:29.152885+0000","last_clean":"2024-04-24T22:40:29.152885+0000","last_became_active":"2024-04-24T22:40:29.152490+0000","last_became_peered":"2024-04-24T22:40:29.152490+0000","last_unstale":"2024-04-24T22:40:29.152885+0000","last_undegraded":"2024-04-24T22:40:29.152885+0000","last_fullsized":"2024-04-24T22:40:29.152885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:54.660889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.150415+0000","last_change":"2024-04-24T22:39:29.661052+0000","last_active":"2024-04-24T22:40:29.150415+0000","last_peered":"2024-04-24T22:40:29.150415+0000","last_clean":"2024-04-24T22:40:29.150415+0000","last_became_active":"2024-04-24T22:39:29.660716+0000","last_became_peered":"2024-04-24T22:39:29.660716+0000","last_unstale":"2024-04-24T22:40:29.150415+0000","last_undegraded":"2024-04-24T22:40:29.150415+0000","last_fullsized":"2024-04-24T22:40:29.150415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:52.016503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.586535+0000","last_change":"2024-04-24T22:39:30.585190+0000","last_active":"2024-04-24T22:40:57.586535+0000","last_peered":"2024-04-24T22:40:57.586535+0000","last_clean":"2024-04-24T22:40:57.586535+0000","last_became_active":"2024-04-24T22:39:30.584846+0000","last_became_peered":"2024-04-24T22:39:30.584846+0000","last_unstale":"2024-04-24T22:40:57.586535+0000","last_undegraded":"2024-04-24T22:40:57.586535+0000","last_fullsized":"2024-04-24T22:40:57.586535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:07.094297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701979+0000","last_change":"2024-04-24T22:39:29.671686+0000","last_active":"2024-04-24T22:40:28.701979+0000","last_peered":"2024-04-24T22:40:28.701979+0000","last_clean":"2024-04-24T22:40:28.701979+0000","last_became_active":"2024-04-24T22:39:29.671338+0000","last_became_peered":"2024-04-24T22:39:29.671338+0000","last_unstale":"2024-04-24T22:40:28.701979+0000","last_undegraded":"2024-04-24T22:40:28.701979+0000","last_fullsized":"2024-04-24T22:40:28.701979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:13.709429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.574445+0000","last_change":"2024-04-24T22:39:30.585560+0000","last_active":"2024-04-24T22:40:57.574445+0000","last_peered":"2024-04-24T22:40:57.574445+0000","last_clean":"2024-04-24T22:40:57.574445+0000","last_became_active":"2024-04-24T22:39:30.585251+0000","last_became_peered":"2024-04-24T22:39:30.585251+0000","last_unstale":"2024-04-24T22:40:57.574445+0000","last_undegraded":"2024-04-24T22:40:57.574445+0000","last_fullsized":"2024-04-24T22:40:57.574445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:29.680556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077985+0000","last_change":"2024-04-24T22:39:30.569105+0000","last_active":"2024-04-24T22:40:29.077985+0000","last_peered":"2024-04-24T22:40:29.077985+0000","last_clean":"2024-04-24T22:40:29.077985+0000","last_became_active":"2024-04-24T22:39:30.566878+0000","last_became_peered":"2024-04-24T22:39:30.566878+0000","last_unstale":"2024-04-24T22:40:29.077985+0000","last_undegraded":"2024-04-24T22:40:29.077985+0000","last_fullsized":"2024-04-24T22:40:29.077985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.334294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":106,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.182473+0000","last_change":"2024-04-24T22:39:29.674334+0000","last_active":"2024-04-24T22:40:57.182473+0000","last_peered":"2024-04-24T22:40:57.182473+0000","last_clean":"2024-04-24T22:40:57.182473+0000","last_became_active":"2024-04-24T22:39:29.673968+0000","last_became_peered":"2024-04-24T22:39:29.673968+0000","last_unstale":"2024-04-24T22:40:57.182473+0000","last_undegraded":"2024-04-24T22:40:57.182473+0000","last_fullsized":"2024-04-24T22:40:57.182473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:10.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.574516+0000","last_change":"2024-04-24T22:39:30.579663+0000","last_active":"2024-04-24T22:40:57.574516+0000","last_peered":"2024-04-24T22:40:57.574516+0000","last_clean":"2024-04-24T22:40:57.574516+0000","last_became_active":"2024-04-24T22:39:30.578730+0000","last_became_peered":"2024-04-24T22:39:30.578730+0000","last_unstale":"2024-04-24T22:40:57.574516+0000","last_undegraded":"2024-04-24T22:40:57.574516+0000","last_fullsized":"2024-04-24T22:40:57.574516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:25.604081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.768758+0000","last_change":"2024-04-24T22:39:29.655316+0000","last_active":"2024-04-24T22:40:28.768758+0000","last_peered":"2024-04-24T22:40:28.768758+0000","last_clean":"2024-04-24T22:40:28.768758+0000","last_became_active":"2024-04-24T22:39:29.654975+0000","last_became_peered":"2024-04-24T22:39:29.654975+0000","last_unstale":"2024-04-24T22:40:28.768758+0000","last_undegraded":"2024-04-24T22:40:28.768758+0000","last_fullsized":"2024-04-24T22:40:28.768758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:20.461744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701776+0000","last_change":"2024-04-24T22:38:09.915368+0000","last_active":"2024-04-24T22:40:28.701776+0000","last_peered":"2024-04-24T22:40:28.701776+0000","last_clean":"2024-04-24T22:40:28.701776+0000","last_became_active":"2024-04-24T22:38:09.914879+0000","last_became_peered":"2024-04-24T22:38:09.914879+0000","last_unstale":"2024-04-24T22:40:28.701776+0000","last_undegraded":"2024-04-24T22:40:28.701776+0000","last_fullsized":"2024-04-24T22:40:28.701776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:34:20.136986+0000","last_clean_scrub_stamp":"2024-04-24T22:34:20.136986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:31.054930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.584106+0000","last_change":"2024-04-24T22:39:30.568169+0000","last_active":"2024-04-24T22:40:57.584106+0000","last_peered":"2024-04-24T22:40:57.584106+0000","last_clean":"2024-04-24T22:40:57.584106+0000","last_became_active":"2024-04-24T22:39:30.567221+0000","last_became_peered":"2024-04-24T22:39:30.567221+0000","last_unstale":"2024-04-24T22:40:57.584106+0000","last_undegraded":"2024-04-24T22:40:57.584106+0000","last_fullsized":"2024-04-24T22:40:57.584106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:06.414726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701859+0000","last_change":"2024-04-24T22:39:29.660378+0000","last_active":"2024-04-24T22:40:28.701859+0000","last_peered":"2024-04-24T22:40:28.701859+0000","last_clean":"2024-04-24T22:40:28.701859+0000","last_became_active":"2024-04-24T22:39:29.660050+0000","last_became_peered":"2024-04-24T22:39:29.660050+0000","last_unstale":"2024-04-24T22:40:28.701859+0000","last_undegraded":"2024-04-24T22:40:28.701859+0000","last_fullsized":"2024-04-24T22:40:28.701859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:06.199365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702124+0000","last_change":"2024-04-24T22:39:29.674925+0000","last_active":"2024-04-24T22:40:28.702124+0000","last_peered":"2024-04-24T22:40:28.702124+0000","last_clean":"2024-04-24T22:40:28.702124+0000","last_became_active":"2024-04-24T22:39:29.674681+0000","last_became_peered":"2024-04-24T22:39:29.674681+0000","last_unstale":"2024-04-24T22:40:28.702124+0000","last_undegraded":"2024-04-24T22:40:28.702124+0000","last_fullsized":"2024-04-24T22:40:28.702124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:04.295567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.595658+0000","last_change":"2024-04-24T22:39:30.581061+0000","last_active":"2024-04-24T22:40:57.595658+0000","last_peered":"2024-04-24T22:40:57.595658+0000","last_clean":"2024-04-24T22:40:57.595658+0000","last_became_active":"2024-04-24T22:39:30.579807+0000","last_became_peered":"2024-04-24T22:39:30.579807+0000","last_unstale":"2024-04-24T22:40:57.595658+0000","last_undegraded":"2024-04-24T22:40:57.595658+0000","last_fullsized":"2024-04-24T22:40:57.595658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:18.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.151213+0000","last_change":"2024-04-24T22:39:29.659948+0000","last_active":"2024-04-24T22:40:29.151213+0000","last_peered":"2024-04-24T22:40:29.151213+0000","last_clean":"2024-04-24T22:40:29.151213+0000","last_became_active":"2024-04-24T22:39:29.659606+0000","last_became_peered":"2024-04-24T22:39:29.659606+0000","last_unstale":"2024-04-24T22:40:29.151213+0000","last_undegraded":"2024-04-24T22:40:29.151213+0000","last_fullsized":"2024-04-24T22:40:29.151213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:32.877670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.586060+0000","last_change":"2024-04-24T22:39:30.587740+0000","last_active":"2024-04-24T22:40:57.586060+0000","last_peered":"2024-04-24T22:40:57.586060+0000","last_clean":"2024-04-24T22:40:57.586060+0000","last_became_active":"2024-04-24T22:39:30.587063+0000","last_became_peered":"2024-04-24T22:39:30.587063+0000","last_unstale":"2024-04-24T22:40:57.586060+0000","last_undegraded":"2024-04-24T22:40:57.586060+0000","last_fullsized":"2024-04-24T22:40:57.586060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:02.704753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.769124+0000","last_change":"2024-04-24T22:39:29.672345+0000","last_active":"2024-04-24T22:40:28.769124+0000","last_peered":"2024-04-24T22:40:28.769124+0000","last_clean":"2024-04-24T22:40:28.769124+0000","last_became_active":"2024-04-24T22:39:29.671957+0000","last_became_peered":"2024-04-24T22:39:29.671957+0000","last_unstale":"2024-04-24T22:40:28.769124+0000","last_undegraded":"2024-04-24T22:40:28.769124+0000","last_fullsized":"2024-04-24T22:40:28.769124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.181431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.588117+0000","last_change":"2024-04-24T22:39:30.595273+0000","last_active":"2024-04-24T22:40:57.588117+0000","last_peered":"2024-04-24T22:40:57.588117+0000","last_clean":"2024-04-24T22:40:57.588117+0000","last_became_active":"2024-04-24T22:39:30.594765+0000","last_became_peered":"2024-04-24T22:39:30.594765+0000","last_unstale":"2024-04-24T22:40:57.588117+0000","last_undegraded":"2024-04-24T22:40:57.588117+0000","last_fullsized":"2024-04-24T22:40:57.588117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.836241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.923654+0000","last_change":"2024-04-24T22:39:29.663779+0000","last_active":"2024-04-24T22:40:28.923654+0000","last_peered":"2024-04-24T22:40:28.923654+0000","last_clean":"2024-04-24T22:40:28.923654+0000","last_became_active":"2024-04-24T22:39:29.663484+0000","last_became_peered":"2024-04-24T22:39:29.663484+0000","last_unstale":"2024-04-24T22:40:28.923654+0000","last_undegraded":"2024-04-24T22:40:28.923654+0000","last_fullsized":"2024-04-24T22:40:28.923654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:37.322047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.544456+0000","last_change":"2024-04-24T22:39:30.568608+0000","last_active":"2024-04-24T22:40:57.544456+0000","last_peered":"2024-04-24T22:40:57.544456+0000","last_clean":"2024-04-24T22:40:57.544456+0000","last_became_active":"2024-04-24T22:39:30.568271+0000","last_became_peered":"2024-04-24T22:39:30.568271+0000","last_unstale":"2024-04-24T22:40:57.544456+0000","last_undegraded":"2024-04-24T22:40:57.544456+0000","last_fullsized":"2024-04-24T22:40:57.544456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:31.433923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.712697+0000","last_change":"2024-04-24T22:39:29.663745+0000","last_active":"2024-04-24T22:40:28.712697+0000","last_peered":"2024-04-24T22:40:28.712697+0000","last_clean":"2024-04-24T22:40:28.712697+0000","last_became_active":"2024-04-24T22:39:29.663436+0000","last_became_peered":"2024-04-24T22:39:29.663436+0000","last_unstale":"2024-04-24T22:40:28.712697+0000","last_undegraded":"2024-04-24T22:40:28.712697+0000","last_fullsized":"2024-04-24T22:40:28.712697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.502177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.703950+0000","last_change":"2024-04-24T22:39:30.580645+0000","last_active":"2024-04-24T22:40:28.703950+0000","last_peered":"2024-04-24T22:40:28.703950+0000","last_clean":"2024-04-24T22:40:28.703950+0000","last_became_active":"2024-04-24T22:39:30.580288+0000","last_became_peered":"2024-04-24T22:39:30.580288+0000","last_unstale":"2024-04-24T22:40:28.703950+0000","last_undegraded":"2024-04-24T22:40:28.703950+0000","last_fullsized":"2024-04-24T22:40:28.703950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:14.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.705162+0000","last_change":"2024-04-24T22:39:29.657571+0000","last_active":"2024-04-24T22:40:28.705162+0000","last_peered":"2024-04-24T22:40:28.705162+0000","last_clean":"2024-04-24T22:40:28.705162+0000","last_became_active":"2024-04-24T22:39:29.657230+0000","last_became_peered":"2024-04-24T22:39:29.657230+0000","last_unstale":"2024-04-24T22:40:28.705162+0000","last_undegraded":"2024-04-24T22:40:28.705162+0000","last_fullsized":"2024-04-24T22:40:28.705162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:26.638528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.593751+0000","last_change":"2024-04-24T22:39:30.590468+0000","last_active":"2024-04-24T22:40:57.593751+0000","last_peered":"2024-04-24T22:40:57.593751+0000","last_clean":"2024-04-24T22:40:57.593751+0000","last_became_active":"2024-04-24T22:39:30.590157+0000","last_became_peered":"2024-04-24T22:39:30.590157+0000","last_unstale":"2024-04-24T22:40:57.593751+0000","last_undegraded":"2024-04-24T22:40:57.593751+0000","last_fullsized":"2024-04-24T22:40:57.593751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.712381+0000","last_change":"2024-04-24T22:40:28.711927+0000","last_active":"2024-04-24T22:40:28.712381+0000","last_peered":"2024-04-24T22:40:28.712381+0000","last_clean":"2024-04-24T22:40:28.712381+0000","last_became_active":"2024-04-24T22:40:28.707475+0000","last_became_peered":"2024-04-24T22:40:28.707475+0000","last_unstale":"2024-04-24T22:40:28.712381+0000","last_undegraded":"2024-04-24T22:40:28.712381+0000","last_fullsized":"2024-04-24T22:40:28.712381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:07.415691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702037+0000","last_change":"2024-04-24T22:39:30.577213+0000","last_active":"2024-04-24T22:40:28.702037+0000","last_peered":"2024-04-24T22:40:28.702037+0000","last_clean":"2024-04-24T22:40:28.702037+0000","last_became_active":"2024-04-24T22:39:30.576883+0000","last_became_peered":"2024-04-24T22:39:30.576883+0000","last_unstale":"2024-04-24T22:40:28.702037+0000","last_undegraded":"2024-04-24T22:40:28.702037+0000","last_fullsized":"2024-04-24T22:40:28.702037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:16.984591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.706753+0000","last_change":"2024-04-24T22:39:29.667082+0000","last_active":"2024-04-24T22:40:28.706753+0000","last_peered":"2024-04-24T22:40:28.706753+0000","last_clean":"2024-04-24T22:40:28.706753+0000","last_became_active":"2024-04-24T22:39:29.666631+0000","last_became_peered":"2024-04-24T22:39:29.666631+0000","last_unstale":"2024-04-24T22:40:28.706753+0000","last_undegraded":"2024-04-24T22:40:28.706753+0000","last_fullsized":"2024-04-24T22:40:28.706753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.001201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702297+0000","last_change":"2024-04-24T22:39:30.565848+0000","last_active":"2024-04-24T22:40:28.702297+0000","last_peered":"2024-04-24T22:40:28.702297+0000","last_clean":"2024-04-24T22:40:28.702297+0000","last_became_active":"2024-04-24T22:39:30.565533+0000","last_became_peered":"2024-04-24T22:39:30.565533+0000","last_unstale":"2024-04-24T22:40:28.702297+0000","last_undegraded":"2024-04-24T22:40:28.702297+0000","last_fullsized":"2024-04-24T22:40:28.702297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:02.406509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.078078+0000","last_change":"2024-04-24T22:39:29.662575+0000","last_active":"2024-04-24T22:40:29.078078+0000","last_peered":"2024-04-24T22:40:29.078078+0000","last_clean":"2024-04-24T22:40:29.078078+0000","last_became_active":"2024-04-24T22:39:29.662273+0000","last_became_peered":"2024-04-24T22:39:29.662273+0000","last_unstale":"2024-04-24T22:40:29.078078+0000","last_undegraded":"2024-04-24T22:40:29.078078+0000","last_fullsized":"2024-04-24T22:40:29.078078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:42.661149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.701861+0000","last_change":"2024-04-24T22:39:30.587813+0000","last_active":"2024-04-24T22:40:28.701861+0000","last_peered":"2024-04-24T22:40:28.701861+0000","last_clean":"2024-04-24T22:40:28.701861+0000","last_became_active":"2024-04-24T22:39:30.587533+0000","last_became_peered":"2024-04-24T22:39:30.587533+0000","last_unstale":"2024-04-24T22:40:28.701861+0000","last_undegraded":"2024-04-24T22:40:28.701861+0000","last_fullsized":"2024-04-24T22:40:28.701861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:27.042695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.707273+0000","last_change":"2024-04-24T22:39:29.664843+0000","last_active":"2024-04-24T22:40:28.707273+0000","last_peered":"2024-04-24T22:40:28.707273+0000","last_clean":"2024-04-24T22:40:28.707273+0000","last_became_active":"2024-04-24T22:39:29.664542+0000","last_became_peered":"2024-04-24T22:39:29.664542+0000","last_unstale":"2024-04-24T22:40:28.707273+0000","last_undegraded":"2024-04-24T22:40:28.707273+0000","last_fullsized":"2024-04-24T22:40:28.707273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:28.635403+0000","last_clean_scrub_stamp":"2024-04-24T22:39:28.635403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:17.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.702207+0000","last_change":"2024-04-24T22:39:30.569753+0000","last_active":"2024-04-24T22:40:28.702207+0000","last_peered":"2024-04-24T22:40:28.702207+0000","last_clean":"2024-04-24T22:40:28.702207+0000","last_became_active":"2024-04-24T22:39:30.569433+0000","last_became_peered":"2024-04-24T22:39:30.569433+0000","last_unstale":"2024-04-24T22:40:28.702207+0000","last_undegraded":"2024-04-24T22:40:28.702207+0000","last_fullsized":"2024-04-24T22:40:28.702207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704027+0000","last_change":"2024-04-24T22:39:30.574076+0000","last_active":"2024-04-24T22:40:28.704027+0000","last_peered":"2024-04-24T22:40:28.704027+0000","last_clean":"2024-04-24T22:40:28.704027+0000","last_became_active":"2024-04-24T22:39:30.573769+0000","last_became_peered":"2024-04-24T22:39:30.573769+0000","last_unstale":"2024-04-24T22:40:28.704027+0000","last_undegraded":"2024-04-24T22:40:28.704027+0000","last_fullsized":"2024-04-24T22:40:28.704027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:55.456402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.559258+0000","last_change":"2024-04-24T22:39:30.570762+0000","last_active":"2024-04-24T22:40:57.559258+0000","last_peered":"2024-04-24T22:40:57.559258+0000","last_clean":"2024-04-24T22:40:57.559258+0000","last_became_active":"2024-04-24T22:39:30.570459+0000","last_became_peered":"2024-04-24T22:39:30.570459+0000","last_unstale":"2024-04-24T22:40:57.559258+0000","last_undegraded":"2024-04-24T22:40:57.559258+0000","last_fullsized":"2024-04-24T22:40:57.559258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:32.826752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.580742+0000","last_change":"2024-04-24T22:39:30.591414+0000","last_active":"2024-04-24T22:40:57.580742+0000","last_peered":"2024-04-24T22:40:57.580742+0000","last_clean":"2024-04-24T22:40:57.580742+0000","last_became_active":"2024-04-24T22:39:30.591135+0000","last_became_peered":"2024-04-24T22:39:30.591135+0000","last_unstale":"2024-04-24T22:40:57.580742+0000","last_undegraded":"2024-04-24T22:40:57.580742+0000","last_fullsized":"2024-04-24T22:40:57.580742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:58.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.592347+0000","last_change":"2024-04-24T22:40:28.715882+0000","last_active":"2024-04-24T22:40:57.592347+0000","last_peered":"2024-04-24T22:40:57.592347+0000","last_clean":"2024-04-24T22:40:57.592347+0000","last_became_active":"2024-04-24T22:40:28.714856+0000","last_became_peered":"2024-04-24T22:40:28.714856+0000","last_unstale":"2024-04-24T22:40:57.592347+0000","last_undegraded":"2024-04-24T22:40:57.592347+0000","last_fullsized":"2024-04-24T22:40:57.592347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.841407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.597537+0000","last_change":"2024-04-24T22:39:30.583955+0000","last_active":"2024-04-24T22:40:57.597537+0000","last_peered":"2024-04-24T22:40:57.597537+0000","last_clean":"2024-04-24T22:40:57.597537+0000","last_became_active":"2024-04-24T22:39:30.583632+0000","last_became_peered":"2024-04-24T22:39:30.583632+0000","last_unstale":"2024-04-24T22:40:57.597537+0000","last_undegraded":"2024-04-24T22:40:57.597537+0000","last_fullsized":"2024-04-24T22:40:57.597537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:44.150140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704151+0000","last_change":"2024-04-24T22:39:30.569989+0000","last_active":"2024-04-24T22:40:28.704151+0000","last_peered":"2024-04-24T22:40:28.704151+0000","last_clean":"2024-04-24T22:40:28.704151+0000","last_became_active":"2024-04-24T22:39:30.569680+0000","last_became_peered":"2024-04-24T22:39:30.569680+0000","last_unstale":"2024-04-24T22:40:28.704151+0000","last_undegraded":"2024-04-24T22:40:28.704151+0000","last_fullsized":"2024-04-24T22:40:28.704151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:55.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.558621+0000","last_change":"2024-04-24T22:39:30.572776+0000","last_active":"2024-04-24T22:40:57.558621+0000","last_peered":"2024-04-24T22:40:57.558621+0000","last_clean":"2024-04-24T22:40:57.558621+0000","last_became_active":"2024-04-24T22:39:30.572445+0000","last_became_peered":"2024-04-24T22:39:30.572445+0000","last_unstale":"2024-04-24T22:40:57.558621+0000","last_undegraded":"2024-04-24T22:40:57.558621+0000","last_fullsized":"2024-04-24T22:40:57.558621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:40.058377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.569444+0000","last_change":"2024-04-24T22:39:30.587453+0000","last_active":"2024-04-24T22:40:57.569444+0000","last_peered":"2024-04-24T22:40:57.569444+0000","last_clean":"2024-04-24T22:40:57.569444+0000","last_became_active":"2024-04-24T22:39:30.586612+0000","last_became_peered":"2024-04-24T22:39:30.586612+0000","last_unstale":"2024-04-24T22:40:57.569444+0000","last_undegraded":"2024-04-24T22:40:57.569444+0000","last_fullsized":"2024-04-24T22:40:57.569444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:24.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:29.077910+0000","last_change":"2024-04-24T22:39:30.592921+0000","last_active":"2024-04-24T22:40:29.077910+0000","last_peered":"2024-04-24T22:40:29.077910+0000","last_clean":"2024-04-24T22:40:29.077910+0000","last_became_active":"2024-04-24T22:39:30.592607+0000","last_became_peered":"2024-04-24T22:39:30.592607+0000","last_unstale":"2024-04-24T22:40:29.077910+0000","last_undegraded":"2024-04-24T22:40:29.077910+0000","last_fullsized":"2024-04-24T22:40:29.077910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:11.240743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.574294+0000","last_change":"2024-04-24T22:39:30.586688+0000","last_active":"2024-04-24T22:40:57.574294+0000","last_peered":"2024-04-24T22:40:57.574294+0000","last_clean":"2024-04-24T22:40:57.574294+0000","last_became_active":"2024-04-24T22:39:30.586259+0000","last_became_peered":"2024-04-24T22:39:30.586259+0000","last_unstale":"2024-04-24T22:40:57.574294+0000","last_undegraded":"2024-04-24T22:40:57.574294+0000","last_fullsized":"2024-04-24T22:40:57.574294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:24.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.589701+0000","last_change":"2024-04-24T22:39:30.591940+0000","last_active":"2024-04-24T22:40:57.589701+0000","last_peered":"2024-04-24T22:40:57.589701+0000","last_clean":"2024-04-24T22:40:57.589701+0000","last_became_active":"2024-04-24T22:39:30.591635+0000","last_became_peered":"2024-04-24T22:39:30.591635+0000","last_unstale":"2024-04-24T22:40:57.589701+0000","last_undegraded":"2024-04-24T22:40:57.589701+0000","last_fullsized":"2024-04-24T22:40:57.589701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:16.097707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:57.595819+0000","last_change":"2024-04-24T22:39:30.588188+0000","last_active":"2024-04-24T22:40:57.595819+0000","last_peered":"2024-04-24T22:40:57.595819+0000","last_clean":"2024-04-24T22:40:57.595819+0000","last_became_active":"2024-04-24T22:39:30.587608+0000","last_became_peered":"2024-04-24T22:39:30.587608+0000","last_unstale":"2024-04-24T22:40:57.595819+0000","last_undegraded":"2024-04-24T22:40:57.595819+0000","last_fullsized":"2024-04-24T22:40:57.595819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:17.690009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-24T22:40:28.704225+0000","last_change":"2024-04-24T22:39:30.567467+0000","last_active":"2024-04-24T22:40:28.704225+0000","last_peered":"2024-04-24T22:40:28.704225+0000","last_clean":"2024-04-24T22:40:28.704225+0000","last_became_active":"2024-04-24T22:39:30.567162+0000","last_became_peered":"2024-04-24T22:39:30.567162+0000","last_unstale":"2024-04-24T22:40:28.704225+0000","last_undegraded":"2024-04-24T22:40:28.704225+0000","last_fullsized":"2024-04-24T22:40:28.704225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:29.639368+0000","last_clean_scrub_stamp":"2024-04-24T22:39:29.639368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:25.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63856,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":528,"num_read_kb":333,"num_write":164,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":456231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1107924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1552384,"data_stored":1048283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":515329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":516533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1564672,"data_stored":1047576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:41:03.938 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T22:41:03.938 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T22:41:03.938 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T22:41:03.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T22:41:04.423 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:04.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:04 smithi111 ceph-mon[27746]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:04.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:04 smithi111 ceph-mon[27746]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:04.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:04 smithi186 ceph-mon[32753]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:05.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:04 smithi186 ceph-mon[32753]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:05.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:04 smithi171 ceph-mon[32337]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:05.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:04 smithi171 ceph-mon[32337]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 -- 172.21.15.111:0/3010630007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c104e10 msgr2=0x7f484c111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 --2- 172.21.15.111:0/3010630007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c104e10 0x7f484c111320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4844009f90 tx=0x7f484400b330 comp rx=0 tx=0).stop 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 -- 172.21.15.111:0/3010630007 shutdown_connections 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 --2- 172.21.15.111:0/3010630007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c104e10 0x7f484c111320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 --2- 172.21.15.111:0/3010630007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c10a4e0 0x7f484c10a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 --2- 172.21.15.111:0/3010630007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 -- 172.21.15.111:0/3010630007 >> 172.21.15.111:0/3010630007 conn(0x7f484c0fff40 msgr2=0x7f484c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:06.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 -- 172.21.15.111:0/3010630007 shutdown_connections 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.485+0000 7f4852e00700 1 -- 172.21.15.111:0/3010630007 wait complete. 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 Processor -- start 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 -- start start 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c104e10 0x7f484c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c10a4e0 0x7f484c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1147b0 con 0x7f484c104450 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f484c114630 con 0x7f484c104e10 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.487+0000 7f4852e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f484c114930 con 0x7f484c10a4e0 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39648/0 (socket says 172.21.15.111:39648) 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 -- 172.21.15.111:0/295518654 learned_addr learned my addr 172.21.15.111:0/295518654 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:06.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f4850e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c10a4e0 0x7f484c06cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484b400700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c104e10 0x7f484c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 -- 172.21.15.111:0/295518654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c10a4e0 msgr2=0x7f484c06cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c10a4e0 0x7f484c06cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 -- 172.21.15.111:0/295518654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c104e10 msgr2=0x7f484c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c104e10 0x7f484c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 -- 172.21.15.111:0/295518654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c06d3e0 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.488+0000 7f484be00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f483c00e9c0 tx=0x7f483c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.489+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00cbd0 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.489+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c06d6d0 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.489+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c00ee50 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.489+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c005680 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.489+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c0716c0 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.490+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484c105b90 con 0x7f484c104450 2024-04-24T22:41:06.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.492+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f483c004750 con 0x7f484c104450 2024-04-24T22:41:06.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.493+0000 7f4848c00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 0x7f48300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.494+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f483c0cf250 con 0x7f484c104450 2024-04-24T22:41:06.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.494+0000 7f484b400700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 0x7f48300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.494+0000 7f484b400700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 0x7f48300a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f484c06e160 tx=0x7f4838009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:06.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.498+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f483c09c160 con 0x7f484c104450 2024-04-24T22:41:06.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.776+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f484c071980 con 0x7f484c104450 2024-04-24T22:41:06.779 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:06 smithi111 ceph-mon[27746]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T22:41:06.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.777+0000 7f4848c00700 1 -- 172.21.15.111:0/295518654 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f483c09ba60 con 0x7f484c104450 2024-04-24T22:41:06.779 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:06.779 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T22:41:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 msgr2=0x7f48300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 0x7f48300a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f484c06e160 tx=0x7f4838009310 comp rx=0 tx=0).stop 2024-04-24T22:41:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 msgr2=0x7f484c073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f483c00e9c0 tx=0x7f483c00ecd0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 shutdown_connections 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f484c10a4e0 0x7f484c06cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f484c104450 0x7f484c073910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f48300a2c20 0x7f48300a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 --2- 172.21.15.111:0/295518654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f484c104e10 0x7f484c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 >> 172.21.15.111:0/295518654 conn(0x7f484c0fff40 msgr2=0x7f484c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.781+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 shutdown_connections 2024-04-24T22:41:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:06.782+0000 7f4852e00700 1 -- 172.21.15.111:0/295518654 wait complete. 2024-04-24T22:41:06.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:06 smithi186 ceph-mon[32753]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T22:41:07.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:06 smithi171 ceph-mon[32337]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T22:41:07.679 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T22:41:07.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.890+0000 7f801d800640 1 -- 172.21.15.111:0/1851623370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80180ffe30 msgr2=0x7f8018100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.890+0000 7f801d800640 1 --2- 172.21.15.111:0/1851623370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80180ffe30 0x7f8018100250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8010009900 tx=0x7f801002a410 comp rx=0 tx=0).stop 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 -- 172.21.15.111:0/1851623370 shutdown_connections 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 --2- 172.21.15.111:0/1851623370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 --2- 172.21.15.111:0/1851623370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018100790 0x7f8018100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 --2- 172.21.15.111:0/1851623370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80180ffe30 0x7f8018100250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 -- 172.21.15.111:0/1851623370 >> 172.21.15.111:0/1851623370 conn(0x7f80180fba10 msgr2=0x7f80180fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 -- 172.21.15.111:0/1851623370 shutdown_connections 2024-04-24T22:41:07.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.891+0000 7f801d800640 1 -- 172.21.15.111:0/1851623370 wait complete. 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.892+0000 7f801d800640 1 Processor -- start 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.892+0000 7f801d800640 1 -- start start 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.892+0000 7f801d800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.892+0000 7f801d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 0x7f8018199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.892+0000 7f801d800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f801d800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f801810f450 con 0x7f8018100790 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f801d800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f801810f2d0 con 0x7f8018101130 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f801d800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f801810f5d0 con 0x7f80180ffe30 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 0x7f8018199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:51770/0 (socket says 172.21.15.111:51770) 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 -- 172.21.15.111:0/3256981082 learned_addr learned my addr 172.21.15.111:0/3256981082 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:07.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f801ce00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 -- 172.21.15.111:0/3256981082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 msgr2=0x7f80181a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 -- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 msgr2=0x7f8018199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 0x7f8018199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017e00640 1 -- 172.21.15.111:0/3256981082 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80181a16c0 con 0x7f80180ffe30 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f801ce00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.893+0000 7f8017400640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 0x7f8018199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f8017e00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8010031440 tx=0x7f8010032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801003f030 con 0x7f80180ffe30 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181a1950 con 0x7f80180ffe30 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010003ce0 con 0x7f80180ffe30 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010035070 con 0x7f80180ffe30 2024-04-24T22:41:07.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.894+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a1e90 con 0x7f80180ffe30 2024-04-24T22:41:07.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.895+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801810ab80 con 0x7f80180ffe30 2024-04-24T22:41:07.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.897+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8010042030 con 0x7f80180ffe30 2024-04-24T22:41:07.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.898+0000 7f8014c00640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 0x7f7ffc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:07.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.899+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f80100ef1d0 con 0x7f80180ffe30 2024-04-24T22:41:07.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.899+0000 7f8017400640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 0x7f7ffc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:07.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.899+0000 7f8017400640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 0x7f7ffc0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f801819afb0 tx=0x7f8000009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:07.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:07.904+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80100bba90 con 0x7f80180ffe30 2024-04-24T22:41:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:07 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/295518654' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T22:41:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:07 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/295518654' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T22:41:08.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.108+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8018062340 con 0x7f80180ffe30 2024-04-24T22:41:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.110+0000 7f8014c00640 1 -- 172.21.15.111:0/3256981082 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7f80100bfd90 con 0x7f80180ffe30 2024-04-24T22:41:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 msgr2=0x7f7ffc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 0x7f7ffc0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f801819afb0 tx=0x7f8000009760 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 msgr2=0x7f8018199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8010031440 tx=0x7f8010032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 shutdown_connections 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f7ffc0a2c10 0x7f7ffc0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8018101130 0x7f80181a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8018100790 0x7f8018199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 --2- 172.21.15.111:0/3256981082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80180ffe30 0x7f8018199aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 >> 172.21.15.111:0/3256981082 conn(0x7f80180fba10 msgr2=0x7f80180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 shutdown_connections 2024-04-24T22:41:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.113+0000 7f801d800640 1 -- 172.21.15.111:0/3256981082 wait complete. 2024-04-24T22:41:08.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:41:08.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:41:08.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:07 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/295518654' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 -- 172.21.15.111:0/2971332269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc102840 msgr2=0x7f86fc102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 --2- 172.21.15.111:0/2971332269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc102840 0x7f86fc102cc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009900 tx=0x7f86e802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 -- 172.21.15.111:0/2971332269 shutdown_connections 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 --2- 172.21.15.111:0/2971332269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc103200 0x7f86fc109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 --2- 172.21.15.111:0/2971332269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc102840 0x7f86fc102cc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 --2- 172.21.15.111:0/2971332269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 -- 172.21.15.111:0/2971332269 >> 172.21.15.111:0/2971332269 conn(0x7f86fc0fcbe0 msgr2=0x7f86fc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 -- 172.21.15.111:0/2971332269 shutdown_connections 2024-04-24T22:41:08.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.340+0000 7f8701200640 1 -- 172.21.15.111:0/2971332269 wait complete. 2024-04-24T22:41:08.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 Processor -- start 2024-04-24T22:41:08.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 -- start start 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc102840 0x7f86fc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc103200 0x7f86fc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d300 con 0x7f86fc101650 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d180 con 0x7f86fc102840 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.341+0000 7f8701200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d480 con 0x7f86fc103200 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39692/0 (socket says 172.21.15.111:39692) 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 -- 172.21.15.111:0/4198402476 learned_addr learned my addr 172.21.15.111:0/4198402476 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fbe00640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc103200 0x7f86fc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86faa00640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc102840 0x7f86fc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 -- 172.21.15.111:0/4198402476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc103200 msgr2=0x7f86fc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc103200 0x7f86fc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 -- 172.21.15.111:0/4198402476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc102840 msgr2=0x7f86fc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc102840 0x7f86fc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 -- 172.21.15.111:0/4198402476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc19f560 con 0x7f86fc101650 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.342+0000 7f86fb400640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9c0 tx=0x7f86ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.343+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00cd80 con 0x7f86fc101650 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.343+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec0035f0 con 0x7f86fc101650 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.343+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec010690 con 0x7f86fc101650 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.343+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc19f850 con 0x7f86fc101650 2024-04-24T22:41:08.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.343+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc19fd70 con 0x7f86fc101650 2024-04-24T22:41:08.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.344+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc108910 con 0x7f86fc101650 2024-04-24T22:41:08.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.346+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86ec0037a0 con 0x7f86fc101650 2024-04-24T22:41:08.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.348+0000 7f86f3e00640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 0x7f86e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.348+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f86ec0cf520 con 0x7f86fc101650 2024-04-24T22:41:08.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.348+0000 7f86faa00640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 0x7f86e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.349+0000 7f86faa00640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 0x7f86e00a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006010 tx=0x7f86e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.353+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86ec09c430 con 0x7f86fc101650 2024-04-24T22:41:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.551+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86fc062340 con 0x7f86fc101650 2024-04-24T22:41:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.552+0000 7f86f3e00640 1 -- 172.21.15.111:0/4198402476 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f86ec09bd30 con 0x7f86fc101650 2024-04-24T22:41:08.554 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:41:08.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.554+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 msgr2=0x7f86e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.554+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 0x7f86e00a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006010 tx=0x7f86e8000bc0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.554+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 msgr2=0x7f86fc197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.554+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9c0 tx=0x7f86ec00ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 shutdown_connections 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f86e00a2b90 0x7f86e00a5050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86fc103200 0x7f86fc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86fc102840 0x7f86fc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 --2- 172.21.15.111:0/4198402476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86fc101650 0x7f86fc197940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 >> 172.21.15.111:0/4198402476 conn(0x7f86fc0fcbe0 msgr2=0x7f86fc105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 shutdown_connections 2024-04-24T22:41:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.555+0000 7f8701200640 1 -- 172.21.15.111:0/4198402476 wait complete. 2024-04-24T22:41:08.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 -- 172.21.15.111:0/2303995923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c105550 msgr2=0x7fb96c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 --2- 172.21.15.111:0/2303995923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c105550 0x7fb96c10bde0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ca0 tx=0x7fb96402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 -- 172.21.15.111:0/2303995923 shutdown_connections 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 --2- 172.21.15.111:0/2303995923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c105550 0x7fb96c10bde0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 --2- 172.21.15.111:0/2303995923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c104b90 0x7fb96c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 --2- 172.21.15.111:0/2303995923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 -- 172.21.15.111:0/2303995923 >> 172.21.15.111:0/2303995923 conn(0x7fb96c0fef30 msgr2=0x7fb96c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:08.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 -- 172.21.15.111:0/2303995923 shutdown_connections 2024-04-24T22:41:08.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.783+0000 7fb970a00640 1 -- 172.21.15.111:0/2303995923 wait complete. 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.784+0000 7fb970a00640 1 Processor -- start 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.784+0000 7fb970a00640 1 -- start start 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c105550 0x7fb96c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10f650 con 0x7fb96c104b90 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10f4d0 con 0x7fb96c105550 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb970a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10f7d0 con 0x7fb96c1039a0 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb961400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb961400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39708/0 (socket says 172.21.15.111:39708) 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb961400640 1 -- 172.21.15.111:0/2482455578 learned_addr learned my addr 172.21.15.111:0/2482455578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb96a800640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c105550 0x7fb96c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 -- 172.21.15.111:0/2482455578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c105550 msgr2=0x7fb96c1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c105550 0x7fb96c1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 -- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 msgr2=0x7fb96c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb969e00640 1 -- 172.21.15.111:0/2482455578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a1890 con 0x7fb96c1039a0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.785+0000 7fb961400640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb969e00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c199c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e9c0 tx=0x7fb95c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00ccc0 con 0x7fb96c1039a0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c0035f0 con 0x7fb96c1039a0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00ce60 con 0x7fb96c1039a0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a1b80 con 0x7fb96c1039a0 2024-04-24T22:41:08.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.786+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a2080 con 0x7fb96c1039a0 2024-04-24T22:41:08.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.787+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c10ac60 con 0x7fb96c1039a0 2024-04-24T22:41:08.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.789+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb95c0037a0 con 0x7fb96c1039a0 2024-04-24T22:41:08.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.790+0000 7fb963400640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 0x7fb9500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.791+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb95c0cf430 con 0x7fb96c1039a0 2024-04-24T22:41:08.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.791+0000 7fb961400640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 0x7fb9500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.791+0000 7fb961400640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 0x7fb9500a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb954011550 tx=0x7fb954005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.795+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb95c09c340 con 0x7fb96c1039a0 2024-04-24T22:41:08.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.995+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb96c062440 con 0x7fb96c1039a0 2024-04-24T22:41:08.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.996+0000 7fb963400640 1 -- 172.21.15.111:0/2482455578 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fb95c09bc40 con 0x7fb96c1039a0 2024-04-24T22:41:08.998 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:41:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:08 smithi186 ceph-mon[32753]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-24T22:41:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:08 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3256981082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:08.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:08 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/4198402476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:09.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 msgr2=0x7fb9500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 0x7fb9500a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb954011550 tx=0x7fb954005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:09.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 msgr2=0x7fb96c199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c199c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e9c0 tx=0x7fb95c00ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 shutdown_connections 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fb9500a2c10 0x7fb9500a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c105550 0x7fb96c1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb96c104b90 0x7fb96c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 --2- 172.21.15.111:0/2482455578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c1039a0 0x7fb96c199c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 >> 172.21.15.111:0/2482455578 conn(0x7fb96c0fef30 msgr2=0x7fb96c107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 shutdown_connections 2024-04-24T22:41:09.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:08.999+0000 7fb970a00640 1 -- 172.21.15.111:0/2482455578 wait complete. 2024-04-24T22:41:09.018 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:08 smithi111 ceph-mon[27746]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-24T22:41:09.018 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:08 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3256981082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:09.018 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:08 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4198402476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:09.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:41:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:08 smithi171 ceph-mon[32337]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-24T22:41:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:08 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3256981082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:09.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:08 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/4198402476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 -- 172.21.15.111:0/419521204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760105430 msgr2=0x7fa76010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 --2- 172.21.15.111:0/419521204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760105430 0x7fa76010bcc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa758009be0 tx=0x7fa75802a6d0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 -- 172.21.15.111:0/419521204 shutdown_connections 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 --2- 172.21.15.111:0/419521204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760105430 0x7fa76010bcc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 --2- 172.21.15.111:0/419521204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760104a70 0x7fa760104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 --2- 172.21.15.111:0/419521204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760103880 0x7fa760103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 -- 172.21.15.111:0/419521204 >> 172.21.15.111:0/419521204 conn(0x7fa7600fedf0 msgr2=0x7fa760101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 -- 172.21.15.111:0/419521204 shutdown_connections 2024-04-24T22:41:09.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.223+0000 7fa765c00640 1 -- 172.21.15.111:0/419521204 wait complete. 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 Processor -- start 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 -- start start 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760104a70 0x7fa76019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760105430 0x7fa7601a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa76010f500 con 0x7fa760103880 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa76010f380 con 0x7fa760105430 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa765c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa76010f680 con 0x7fa760104a70 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39730/0 (socket says 172.21.15.111:39730) 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.224+0000 7fa75fe00640 1 -- 172.21.15.111:0/2061616603 learned_addr learned my addr 172.21.15.111:0/2061616603 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:09.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa757e00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760104a70 0x7fa76019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa765200640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760105430 0x7fa7601a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 -- 172.21.15.111:0/2061616603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760104a70 msgr2=0x7fa76019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760104a70 0x7fa76019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 -- 172.21.15.111:0/2061616603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760105430 msgr2=0x7fa7601a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760105430 0x7fa7601a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 -- 172.21.15.111:0/2061616603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7601a1780 con 0x7fa760103880 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75fe00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75000ccc0 con 0x7fa760103880 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7500035f0 con 0x7fa760103880 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75000ce60 con 0x7fa760103880 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.225+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7601a1a70 con 0x7fa760103880 2024-04-24T22:41:09.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.226+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a1f70 con 0x7fa760103880 2024-04-24T22:41:09.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.227+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76010ab40 con 0x7fa760103880 2024-04-24T22:41:09.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.229+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa750003b10 con 0x7fa760103880 2024-04-24T22:41:09.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.230+0000 7fa75d600640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 0x7fa7440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.230+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa7500cf2d0 con 0x7fa760103880 2024-04-24T22:41:09.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.230+0000 7fa757e00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 0x7fa7440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.231+0000 7fa757e00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 0x7fa7440a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa76019b070 tx=0x7fa748005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:09.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.234+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa75009c1e0 con 0x7fa760103880 2024-04-24T22:41:09.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.435+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa76002d3d0 con 0x7fa760103880 2024-04-24T22:41:09.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.436+0000 7fa75d600640 1 -- 172.21.15.111:0/2061616603 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fa75009bae0 con 0x7fa760103880 2024-04-24T22:41:09.438 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.438+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 msgr2=0x7fa7440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.438+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 0x7fa7440a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa76019b070 tx=0x7fa748005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.438+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 msgr2=0x7fa760199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.438+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000eeb0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 shutdown_connections 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa7440a2c10 0x7fa7440a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa760105430 0x7fa7601a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa760104a70 0x7fa76019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 --2- 172.21.15.111:0/2061616603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa760103880 0x7fa760199b60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 >> 172.21.15.111:0/2061616603 conn(0x7fa7600fedf0 msgr2=0x7fa760107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 shutdown_connections 2024-04-24T22:41:09.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.439+0000 7fa765c00640 1 -- 172.21.15.111:0/2061616603 wait complete. 2024-04-24T22:41:09.460 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:41:09.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 -- 172.21.15.111:0/1904495245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c0102040 msgr2=0x7f46c0104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 --2- 172.21.15.111:0/1904495245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c0102040 0x7f46c0104430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009900 tx=0x7f46b002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 -- 172.21.15.111:0/1904495245 shutdown_connections 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 --2- 172.21.15.111:0/1904495245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c01072a0 0x7f46c01096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 --2- 172.21.15.111:0/1904495245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0104970 0x7f46c0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 --2- 172.21.15.111:0/1904495245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c0102040 0x7f46c0104430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 -- 172.21.15.111:0/1904495245 >> 172.21.15.111:0/1904495245 conn(0x7f46c00fba10 msgr2=0x7f46c00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 -- 172.21.15.111:0/1904495245 shutdown_connections 2024-04-24T22:41:09.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.660+0000 7f46c4c00640 1 -- 172.21.15.111:0/1904495245 wait complete. 2024-04-24T22:41:09.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.661+0000 7f46c4c00640 1 Processor -- start 2024-04-24T22:41:09.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 -- start start 2024-04-24T22:41:09.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0102040 0x7f46c01978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 0x7f46c019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f46c010d280 con 0x7f46c0104970 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46c010d100 con 0x7f46c0102040 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46c4c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46c010d400 con 0x7f46c01072a0 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39748/0 (socket says 172.21.15.111:39748) 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46bf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0102040 0x7f46c01978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.662+0000 7f46b6a00640 1 -- 172.21.15.111:0/3518993909 learned_addr learned my addr 172.21.15.111:0/3518993909 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46bfe00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 0x7f46c019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 -- 172.21.15.111:0/3518993909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 msgr2=0x7f46c019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 0x7f46c019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 -- 172.21.15.111:0/3518993909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0102040 msgr2=0x7f46c01978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0102040 0x7f46c01978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 -- 172.21.15.111:0/3518993909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c019f510 con 0x7f46c0104970 2024-04-24T22:41:09.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46bfe00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 0x7f46c019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:09.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46b6a00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9c0 tx=0x7f46a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:09.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a800cd80 con 0x7f46c0104970 2024-04-24T22:41:09.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8005500 con 0x7f46c0104970 2024-04-24T22:41:09.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8010690 con 0x7f46c0104970 2024-04-24T22:41:09.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.663+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c019f800 con 0x7f46c0104970 2024-04-24T22:41:09.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.664+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c019fda0 con 0x7f46c0104970 2024-04-24T22:41:09.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.665+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c00fd500 con 0x7f46c0104970 2024-04-24T22:41:09.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.667+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f46a80046c0 con 0x7f46c0104970 2024-04-24T22:41:09.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.668+0000 7f46bcc00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 0x7f46a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:09.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.668+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f46a80cf460 con 0x7f46c0104970 2024-04-24T22:41:09.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.668+0000 7f46bf400640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 0x7f46a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:09.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.669+0000 7f46bf400640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 0x7f46a40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f46b0031440 tx=0x7f46b00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:09.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.672+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46a809c370 con 0x7f46c0104970 2024-04-24T22:41:09.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.916+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46c002d170 con 0x7f46c0104970 2024-04-24T22:41:09.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.918+0000 7f46bcc00640 1 -- 172.21.15.111:0/3518993909 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7f46a809bc70 con 0x7f46c0104970 2024-04-24T22:41:09.920 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:09.920 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:41:09.921 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 msgr2=0x7f46a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 0x7f46a40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f46b0031440 tx=0x7f46b00315b0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 msgr2=0x7f46c0197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9c0 tx=0x7f46a800ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 shutdown_connections 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f46a40a2c10 0x7f46a40a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c01072a0 0x7f46c019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46c0104970 0x7f46c0197e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 --2- 172.21.15.111:0/3518993909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46c0102040 0x7f46c01978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 >> 172.21.15.111:0/3518993909 conn(0x7f46c00fba10 msgr2=0x7f46c01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 shutdown_connections 2024-04-24T22:41:09.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:09.921+0000 7f46c4c00640 1 -- 172.21.15.111:0/3518993909 wait complete. 2024-04-24T22:41:09.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:09 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2482455578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:09 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2061616603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.011 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:09 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2482455578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.012 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:09 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2061616603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 -- 172.21.15.111:0/3970573441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 msgr2=0x7f0fac1101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/3970573441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1101e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009be0 tx=0x7f0fa402a6d0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 -- 172.21.15.111:0/3970573441 shutdown_connections 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/3970573441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1101e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/3970573441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/3970573441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 -- 172.21.15.111:0/3970573441 >> 172.21.15.111:0/3970573441 conn(0x7f0fac076d60 msgr2=0x7f0fac079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 -- 172.21.15.111:0/3970573441 shutdown_connections 2024-04-24T22:41:10.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.145+0000 7f0fb0e00640 1 -- 172.21.15.111:0/3970573441 wait complete. 2024-04-24T22:41:10.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.146+0000 7f0fb0e00640 1 Processor -- start 2024-04-24T22:41:10.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.146+0000 7f0fb0e00640 1 -- start start 2024-04-24T22:41:10.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac113a20 con 0x7f0fac108f90 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1138a0 con 0x7f0fac107da0 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fb0e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac113ba0 con 0x7f0fac109950 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fa9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fa9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fa9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44142/0 (socket says 172.21.15.111:44142) 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0fa9e00640 1 -- 172.21.15.111:0/350953516 learned_addr learned my addr 172.21.15.111:0/350953516 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.147+0000 7f0faa800640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 -- 172.21.15.111:0/350953516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 msgr2=0x7f0fac1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 -- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 msgr2=0x7f0fac19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 -- 172.21.15.111:0/350953516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac1a5c50 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9400640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa9e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400e9c0 tx=0x7f0f9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400cd80 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac1a5f40 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f940035f0 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.148+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94010690 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.149+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a63a0 con 0x7f0fac107da0 2024-04-24T22:41:10.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.150+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac10f060 con 0x7f0fac107da0 2024-04-24T22:41:10.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.152+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0f940037a0 con 0x7f0fac107da0 2024-04-24T22:41:10.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.153+0000 7f0fa2a00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 0x7f0f900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.153+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0f940cf420 con 0x7f0fac107da0 2024-04-24T22:41:10.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.153+0000 7f0fa9400640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 0x7f0f900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.154+0000 7f0fa9400640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 0x7f0f900a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0096c0 tx=0x7f0f9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:09 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2482455578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:09 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2061616603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:10.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.155+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f9409c330 con 0x7f0fac107da0 2024-04-24T22:41:10.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.356+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fac02d2f0 con 0x7f0fac107da0 2024-04-24T22:41:10.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.358+0000 7f0fa2a00640 1 -- 172.21.15.111:0/350953516 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7f0f9409bc30 con 0x7f0fac107da0 2024-04-24T22:41:10.360 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:10.361 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 msgr2=0x7f0f900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 0x7f0f900a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0096c0 tx=0x7f0f9c009260 comp rx=0 tx=0).stop 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 msgr2=0x7f0fac19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400e9c0 tx=0x7f0f9400ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 shutdown_connections 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f900a2b90 0x7f0f900a5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fac109950 0x7f0fac1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fac108f90 0x7f0fac19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 --2- 172.21.15.111:0/350953516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fac107da0 0x7f0fac19e030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 >> 172.21.15.111:0/350953516 conn(0x7f0fac076d60 msgr2=0x7f0fac10c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:10.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 shutdown_connections 2024-04-24T22:41:10.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.361+0000 7f0fb0e00640 1 -- 172.21.15.111:0/350953516 wait complete. 2024-04-24T22:41:10.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:41:10.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:41:10.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 -- 172.21.15.111:0/521630262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44481016d0 msgr2=0x7f4448101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 --2- 172.21.15.111:0/521630262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44481016d0 0x7f4448101af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4440009ca0 tx=0x7f444002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 -- 172.21.15.111:0/521630262 shutdown_connections 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 --2- 172.21.15.111:0/521630262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448103280 0x7f4448109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 --2- 172.21.15.111:0/521630262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481028c0 0x7f4448102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 --2- 172.21.15.111:0/521630262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44481016d0 0x7f4448101af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 -- 172.21.15.111:0/521630262 >> 172.21.15.111:0/521630262 conn(0x7f44480fcc40 msgr2=0x7f44480ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 -- 172.21.15.111:0/521630262 shutdown_connections 2024-04-24T22:41:10.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.584+0000 7f4447e00640 1 -- 172.21.15.111:0/521630262 wait complete. 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.585+0000 7f4447e00640 1 Processor -- start 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 -- start start 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44481028c0 0x7f4448197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4448103280 0x7f444819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f444810d330 con 0x7f44481028c0 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f444810d1b0 con 0x7f44481016d0 2024-04-24T22:41:10.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f444810d4b0 con 0x7f4448103280 2024-04-24T22:41:10.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4446a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4446a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44164/0 (socket says 172.21.15.111:44164) 2024-04-24T22:41:10.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4446a00640 1 -- 172.21.15.111:0/2229140263 learned_addr learned my addr 172.21.15.111:0/2229140263 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:10.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4447400640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4448103280 0x7f444819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.586+0000 7f4446000640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44481028c0 0x7f4448197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 -- 172.21.15.111:0/2229140263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4448103280 msgr2=0x7f444819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4448103280 0x7f444819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 -- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44481028c0 msgr2=0x7f4448197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44481028c0 0x7f4448197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 -- 172.21.15.111:0/2229140263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444819f620 con 0x7f44481016d0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.587+0000 7f4446a00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4440000f80 tx=0x7f44400319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.588+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440003bf0 con 0x7f44481016d0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.588+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4440003d90 con 0x7f44481016d0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.588+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4448073970 con 0x7f44481016d0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.588+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444002fd50 con 0x7f44481016d0 2024-04-24T22:41:10.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.588+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4448073f10 con 0x7f44481016d0 2024-04-24T22:41:10.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.589+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4448108a40 con 0x7f44481016d0 2024-04-24T22:41:10.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.591+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4440004b40 con 0x7f44481016d0 2024-04-24T22:41:10.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.592+0000 7f443f400640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 0x7f442c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:10.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.592+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f44400f05d0 con 0x7f44481016d0 2024-04-24T22:41:10.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.592+0000 7f4446000640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 0x7f442c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:10.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.593+0000 7f4446000640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 0x7f442c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4448198f10 tx=0x7f4430012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:10.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.595+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f44400bd570 con 0x7f44481016d0 2024-04-24T22:41:10.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.795+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44480752f0 con 0x7f44481016d0 2024-04-24T22:41:10.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.796+0000 7f443f400640 1 -- 172.21.15.111:0/2229140263 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f44400bce70 con 0x7f44481016d0 2024-04-24T22:41:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 msgr2=0x7f442c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 0x7f442c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4448198f10 tx=0x7f4430012040 comp rx=0 tx=0).stop 2024-04-24T22:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 msgr2=0x7f4448197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4440000f80 tx=0x7f44400319f0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 shutdown_connections 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f442c0a2c10 0x7f442c0a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4448103280 0x7f444819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44481028c0 0x7f4448197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 --2- 172.21.15.111:0/2229140263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44481016d0 0x7f4448197a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 >> 172.21.15.111:0/2229140263 conn(0x7f44480fcc40 msgr2=0x7f4448105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 shutdown_connections 2024-04-24T22:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:10.799+0000 7f4447e00640 1 -- 172.21.15.111:0/2229140263 wait complete. 2024-04-24T22:41:10.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:41:10.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:10 smithi186 ceph-mon[32753]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:11.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:10 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3518993909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:10 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/350953516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.010 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:10 smithi111 ceph-mon[27746]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:11.010 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:10 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3518993909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.010 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:10 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/350953516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:10 smithi171 ceph-mon[32337]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T22:41:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:10 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3518993909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:10 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/350953516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.164+0000 7fd045c00640 1 -- 172.21.15.111:0/119235784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 msgr2=0x7fd040105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.164+0000 7fd045c00640 1 --2- 172.21.15.111:0/119235784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd040105010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802a410 comp rx=0 tx=0).stop 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 -- 172.21.15.111:0/119235784 shutdown_connections 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 --2- 172.21.15.111:0/119235784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd040105550 0x7fd04010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 --2- 172.21.15.111:0/119235784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd040105010 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 --2- 172.21.15.111:0/119235784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0401039a0 0x7fd040103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 -- 172.21.15.111:0/119235784 >> 172.21.15.111:0/119235784 conn(0x7fd0400fef30 msgr2=0x7fd040101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 -- 172.21.15.111:0/119235784 shutdown_connections 2024-04-24T22:41:11.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.165+0000 7fd045c00640 1 -- 172.21.15.111:0/119235784 wait complete. 2024-04-24T22:41:11.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 Processor -- start 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 -- start start 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 0x7fd040199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 0x7fd0401a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010f670 con 0x7fd040104b90 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010f4f0 con 0x7fd0401039a0 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd045c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010f7f0 con 0x7fd040105550 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd036000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd036000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39786/0 (socket says 172.21.15.111:39786) 2024-04-24T22:41:11.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd036000640 1 -- 172.21.15.111:0/868964189 learned_addr learned my addr 172.21.15.111:0/868964189 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd03ea00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 0x7fd040199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.166+0000 7fd03f400640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 0x7fd0401a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 -- 172.21.15.111:0/868964189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 msgr2=0x7fd040199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 0x7fd040199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 -- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 msgr2=0x7fd04019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 -- 172.21.15.111:0/868964189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0401a1940 con 0x7fd040105550 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd036000640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03ea00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 0x7fd040199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.167+0000 7fd03f400640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 0x7fd0401a12a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd03800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.168+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800ce20 con 0x7fd040105550 2024-04-24T22:41:11.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.168+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0401a1c30 con 0x7fd040105550 2024-04-24T22:41:11.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.168+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038005020 con 0x7fd040105550 2024-04-24T22:41:11.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.168+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038010720 con 0x7fd040105550 2024-04-24T22:41:11.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.168+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a2180 con 0x7fd040105550 2024-04-24T22:41:11.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.169+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04010acc0 con 0x7fd040105550 2024-04-24T22:41:11.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.171+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd038005540 con 0x7fd040105550 2024-04-24T22:41:11.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.172+0000 7fd037e00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 0x7fd0240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.173+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0380cf930 con 0x7fd040105550 2024-04-24T22:41:11.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.173+0000 7fd03ea00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 0x7fd0240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.173+0000 7fd03ea00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 0x7fd0240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd04019b200 tx=0x7fd030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.174+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd03809c840 con 0x7fd040105550 2024-04-24T22:41:11.378 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.376+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd04002d170 con 0x7fd040105550 2024-04-24T22:41:11.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.377+0000 7fd037e00640 1 -- 172.21.15.111:0/868964189 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd03809c140 con 0x7fd040105550 2024-04-24T22:41:11.379 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:41:11.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 msgr2=0x7fd0240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 0x7fd0240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd04019b200 tx=0x7fd030005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:11.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 msgr2=0x7fd0401a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 0x7fd0401a12a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd03800ef10 comp rx=0 tx=0).stop 2024-04-24T22:41:11.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 shutdown_connections 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fd0240a2b90 0x7fd0240a5050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd040105550 0x7fd0401a12a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd040104b90 0x7fd04019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 --2- 172.21.15.111:0/868964189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0401039a0 0x7fd040199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.380+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 >> 172.21.15.111:0/868964189 conn(0x7fd0400fef30 msgr2=0x7fd040107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.381+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 shutdown_connections 2024-04-24T22:41:11.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.381+0000 7fd045c00640 1 -- 172.21.15.111:0/868964189 wait complete. 2024-04-24T22:41:11.401 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:41:11.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 -- 172.21.15.111:0/1777941519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 msgr2=0x7f2eb01044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/1777941519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb01044c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea002a400 comp rx=0 tx=0).stop 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 -- 172.21.15.111:0/1777941519 shutdown_connections 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/1777941519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 0x7f2eb0109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/1777941519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb0106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/1777941519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb01044c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.604+0000 7f2eb5c00640 1 -- 172.21.15.111:0/1777941519 >> 172.21.15.111:0/1777941519 conn(0x7f2eb00fbae0 msgr2=0x7f2eb00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.605+0000 7f2eb5c00640 1 -- 172.21.15.111:0/1777941519 shutdown_connections 2024-04-24T22:41:11.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.605+0000 7f2eb5c00640 1 -- 172.21.15.111:0/1777941519 wait complete. 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 Processor -- start 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 -- start start 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 0x7f2eb01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010d340 con 0x7f2eb0104a00 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010d1c0 con 0x7f2eb01020d0 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eb5c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010d4c0 con 0x7f2eb0107330 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39808/0 (socket says 172.21.15.111:39808) 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2ea7400640 1 -- 172.21.15.111:0/2139459035 learned_addr learned my addr 172.21.15.111:0/2139459035 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.606+0000 7f2eafe00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2eb5200640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 0x7f2eb01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 -- 172.21.15.111:0/2139459035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 msgr2=0x7f2eb01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 0x7f2eb01a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 -- 172.21.15.111:0/2139459035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 msgr2=0x7f2eb0199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb0199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 -- 172.21.15.111:0/2139459035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb01a1790 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ea7400640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9c0 tx=0x7f2e9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800cd80 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e98005500 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb01a1a80 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.607+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98010690 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.608+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a1fd0 con 0x7f2eb0104a00 2024-04-24T22:41:11.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.609+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb0108e80 con 0x7f2eb0104a00 2024-04-24T22:41:11.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.611+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2e980046c0 con 0x7f2eb0104a00 2024-04-24T22:41:11.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.612+0000 7f2ead600640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 0x7f2e940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.612+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2e980cf430 con 0x7f2eb0104a00 2024-04-24T22:41:11.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.612+0000 7f2eafe00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 0x7f2e940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.613+0000 7f2eafe00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 0x7f2e940a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0031440 tx=0x7f2ea00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.616+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2e9809c340 con 0x7f2eb0104a00 2024-04-24T22:41:11.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.816+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2eb0062340 con 0x7f2eb0104a00 2024-04-24T22:41:11.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.817+0000 7f2ead600640 1 -- 172.21.15.111:0/2139459035 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f2e9809bc40 con 0x7f2eb0104a00 2024-04-24T22:41:11.819 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:41:11.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 msgr2=0x7f2e940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 0x7f2e940a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0031440 tx=0x7f2ea00315b0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 msgr2=0x7f2eb019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9c0 tx=0x7f2e9800ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 shutdown_connections 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2e940a2b90 0x7f2e940a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb0107330 0x7f2eb01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb0104a00 0x7f2eb019a0b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 --2- 172.21.15.111:0/2139459035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eb01020d0 0x7f2eb0199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 >> 172.21.15.111:0/2139459035 conn(0x7f2eb00fbae0 msgr2=0x7f2eb0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.819+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 shutdown_connections 2024-04-24T22:41:11.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:11.820+0000 7f2eb5c00640 1 -- 172.21.15.111:0/2139459035 wait complete. 2024-04-24T22:41:11.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:41:11.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2229140263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/868964189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2229140263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/868964189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 -- 172.21.15.111:0/355759523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960102970 msgr2=0x7f2960104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 --2- 172.21.15.111:0/355759523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960102970 0x7f2960104de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f294c009900 tx=0x7f294c02a410 comp rx=0 tx=0).stop 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 -- 172.21.15.111:0/355759523 shutdown_connections 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 --2- 172.21.15.111:0/355759523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960105320 0x7f29601057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 --2- 172.21.15.111:0/355759523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960102970 0x7f2960104de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 --2- 172.21.15.111:0/355759523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f2960102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 -- 172.21.15.111:0/355759523 >> 172.21.15.111:0/355759523 conn(0x7f29600fbc40 msgr2=0x7f29600fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 -- 172.21.15.111:0/355759523 shutdown_connections 2024-04-24T22:41:12.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.036+0000 7f2967200640 1 -- 172.21.15.111:0/355759523 wait complete. 2024-04-24T22:41:12.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 Processor -- start 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 -- start start 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f296019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 0x7f296019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2960113a50 con 0x7f2960105320 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f29601138d0 con 0x7f2960100000 2024-04-24T22:41:12.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2967200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2960113bd0 con 0x7f2960102970 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2965200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f295fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f296019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2965200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39822/0 (socket says 172.21.15.111:39822) 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.037+0000 7f2965200640 1 -- 172.21.15.111:0/3053489453 learned_addr learned my addr 172.21.15.111:0/3053489453 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295f400640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 0x7f296019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 -- 172.21.15.111:0/3053489453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 msgr2=0x7f296019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 0x7f296019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 -- 172.21.15.111:0/3053489453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 msgr2=0x7f296019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f296019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 -- 172.21.15.111:0/3053489453 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29601a5c60 con 0x7f2960105320 2024-04-24T22:41:12.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295f400640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 0x7f296019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295fe00640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f296019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f2965200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f295800eb10 tx=0x7f295800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958004b20 con 0x7f2960105320 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2958004cc0 con 0x7f2960105320 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.038+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958010760 con 0x7f2960105320 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.039+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29601a5f50 con 0x7f2960105320 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.039+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601a6360 con 0x7f2960105320 2024-04-24T22:41:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.040+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29600fd730 con 0x7f2960105320 2024-04-24T22:41:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.042+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2958005020 con 0x7f2960105320 2024-04-24T22:41:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.043+0000 7f295cc00640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 0x7f29440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.043+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f29580cf7e0 con 0x7f2960105320 2024-04-24T22:41:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.043+0000 7f295fe00640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 0x7f29440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.044+0000 7f295fe00640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 0x7f29440a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2950009fc0 tx=0x7f2950005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.048+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f295809c6f0 con 0x7f2960105320 2024-04-24T22:41:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2229140263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/868964189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.272+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f296002d170 con 0x7f2960105320 2024-04-24T22:41:12.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.274+0000 7f295cc00640 1 -- 172.21.15.111:0/3053489453 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7f295809bff0 con 0x7f2960105320 2024-04-24T22:41:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:12.276 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:41:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 msgr2=0x7f29440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 0x7f29440a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2950009fc0 tx=0x7f2950005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 msgr2=0x7f29601a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f295800eb10 tx=0x7f295800c2d0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 shutdown_connections 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f29440a2b90 0x7f29440a5050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960105320 0x7f29601a55f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2960102970 0x7f296019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 --2- 172.21.15.111:0/3053489453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2960100000 0x7f296019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 >> 172.21.15.111:0/3053489453 conn(0x7f29600fbc40 msgr2=0x7f2960100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 shutdown_connections 2024-04-24T22:41:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.278+0000 7f2967200640 1 -- 172.21.15.111:0/3053489453 wait complete. 2024-04-24T22:41:12.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 -- 172.21.15.111:0/3607991550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101049b0 msgr2=0x7fbf10104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/3607991550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101049b0 0x7fbf10104e30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009860 tx=0x7fbefc02a340 comp rx=0 tx=0).stop 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 -- 172.21.15.111:0/3607991550 shutdown_connections 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/3607991550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf10105370 0x7fbf1010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/3607991550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101049b0 0x7fbf10104e30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/3607991550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf101037c0 0x7fbf10103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 -- 172.21.15.111:0/3607991550 >> 172.21.15.111:0/3607991550 conn(0x7fbf100fed50 msgr2=0x7fbf10101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 -- 172.21.15.111:0/3607991550 shutdown_connections 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.495+0000 7fbf0fe00640 1 -- 172.21.15.111:0/3607991550 wait complete. 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 Processor -- start 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 -- start start 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101037c0 0x7fbf10199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 0x7fbf10199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010f440 con 0x7fbf10105370 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010f2c0 con 0x7fbf101049b0 2024-04-24T22:41:12.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.496+0000 7fbf0fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010f5c0 con 0x7fbf101037c0 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39844/0 (socket says 172.21.15.111:39844) 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 -- 172.21.15.111:0/2846425330 learned_addr learned my addr 172.21.15.111:0/2846425330 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0e000640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 0x7fbf10199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0ea00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101037c0 0x7fbf10199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 -- 172.21.15.111:0/2846425330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101037c0 msgr2=0x7fbf10199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101037c0 0x7fbf10199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 -- 172.21.15.111:0/2846425330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 msgr2=0x7fbf10199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 0x7fbf10199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 -- 172.21.15.111:0/2846425330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf101a16d0 con 0x7fbf10105370 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0e000640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 0x7fbf10199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.497+0000 7fbf0f400640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800eb10 tx=0x7fbf0800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.498+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800ce20 con 0x7fbf10105370 2024-04-24T22:41:12.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.498+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101a19c0 con 0x7fbf10105370 2024-04-24T22:41:12.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.498+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf08005020 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.498+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08010690 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.498+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a1ee0 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.499+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1010aae0 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.501+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbf08005540 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.502+0000 7fbf07400640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 0x7fbef40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.502+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbf080cf770 con 0x7fbf10105370 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.502+0000 7fbf0ea00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 0x7fbef40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.503+0000 7fbf0ea00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 0x7fbef40a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf10104810 tx=0x7fbf00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.506+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbf0809c680 con 0x7fbf10105370 2024-04-24T22:41:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.707+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf1002d170 con 0x7fbf10105370 2024-04-24T22:41:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.711+0000 7fbf07400640 1 -- 172.21.15.111:0/2846425330 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7fbf0809bf80 con 0x7fbf10105370 2024-04-24T22:41:12.713 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:12.713 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 msgr2=0x7fbef40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 0x7fbef40a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf10104810 tx=0x7fbf00009260 comp rx=0 tx=0).stop 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 msgr2=0x7fbf101a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800eb10 tx=0x7fbf0800ef10 comp rx=0 tx=0).stop 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 shutdown_connections 2024-04-24T22:41:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fbef40a2b90 0x7fbef40a5050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10105370 0x7fbf101a1060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf101049b0 0x7fbf10199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 --2- 172.21.15.111:0/2846425330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf101037c0 0x7fbf10199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.713+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 >> 172.21.15.111:0/2846425330 conn(0x7fbf100fed50 msgr2=0x7fbf10107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.714+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 shutdown_connections 2024-04-24T22:41:12.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.714+0000 7fbf0fe00640 1 -- 172.21.15.111:0/2846425330 wait complete. 2024-04-24T22:41:12.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:41:12.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 -- 172.21.15.111:0/2502998500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c103160 msgr2=0x7ff75c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 --2- 172.21.15.111:0/2502998500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c103160 0x7ff75c1099f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff754009ca0 tx=0x7ff75402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 -- 172.21.15.111:0/2502998500 shutdown_connections 2024-04-24T22:41:12.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 --2- 172.21.15.111:0/2502998500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c103160 0x7ff75c1099f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 --2- 172.21.15.111:0/2502998500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff75c1027a0 0x7ff75c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 --2- 172.21.15.111:0/2502998500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 0x7ff75c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 -- 172.21.15.111:0/2502998500 >> 172.21.15.111:0/2502998500 conn(0x7ff75c0fcb20 msgr2=0x7ff75c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 -- 172.21.15.111:0/2502998500 shutdown_connections 2024-04-24T22:41:12.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.926+0000 7ff75be00640 1 -- 172.21.15.111:0/2502998500 wait complete. 2024-04-24T22:41:12.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.927+0000 7ff75be00640 1 Processor -- start 2024-04-24T22:41:12.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.927+0000 7ff75be00640 1 -- start start 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.927+0000 7ff75be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 0x7ff75c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.927+0000 7ff75be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff75be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff75c103160 0x7ff75c1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff75be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c10d280 con 0x7ff75c1027a0 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff75be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c10d100 con 0x7ff75c103160 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff75be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c10d400 con 0x7ff75c1015b0 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39872/0 (socket says 172.21.15.111:39872) 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 -- 172.21.15.111:0/3248182720 learned_addr learned my addr 172.21.15.111:0/3248182720 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff75aa00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 0x7ff75c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 -- 172.21.15.111:0/3248182720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 msgr2=0x7ff75c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 0x7ff75c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 -- 172.21.15.111:0/3248182720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff75c103160 msgr2=0x7ff75c1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff75c103160 0x7ff75c1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 -- 172.21.15.111:0/3248182720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c1a16d0 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.928+0000 7ff752000640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e9f0 tx=0x7ff74400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.929+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400cd80 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.929+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744010910 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.929+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c1a19c0 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.929+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744018990 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.929+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a1f60 con 0x7ff75c1027a0 2024-04-24T22:41:12.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.930+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c1a2250 con 0x7ff75c1027a0 2024-04-24T22:41:12.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.932+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff744010ac0 con 0x7ff75c1027a0 2024-04-24T22:41:12.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.933+0000 7ff753e00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 0x7ff7400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.933+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff7440cf3a0 con 0x7ff75c1027a0 2024-04-24T22:41:12.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.933+0000 7ff75aa00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 0x7ff7400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.934+0000 7ff75aa00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 0x7ff7400a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff74c005fd0 tx=0x7ff74c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:12.937+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff74409c2b0 con 0x7ff75c1027a0 2024-04-24T22:41:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:12 smithi186 ceph-mon[32753]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:41:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2139459035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3053489453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:12.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2846425330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:12 smithi111 ceph-mon[27746]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:41:13.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2139459035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:13.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3053489453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2846425330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:12 smithi171 ceph-mon[32337]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:41:13.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2139459035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:13.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3053489453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2846425330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.178+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff75c1a2420 con 0x7ff75c1027a0 2024-04-24T22:41:13.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.180+0000 7ff753e00640 1 -- 172.21.15.111:0/3248182720 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7ff74409bbb0 con 0x7ff75c1027a0 2024-04-24T22:41:13.187 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:13.188 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 msgr2=0x7ff7400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 0x7ff7400a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff74c005fd0 tx=0x7ff74c00b390 comp rx=0 tx=0).stop 2024-04-24T22:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 msgr2=0x7ff75c199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e9f0 tx=0x7ff74400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 shutdown_connections 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ff7400a2c10 0x7ff7400a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff75c103160 0x7ff75c1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff75c1027a0 0x7ff75c199ff0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 --2- 172.21.15.111:0/3248182720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff75c1015b0 0x7ff75c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 >> 172.21.15.111:0/3248182720 conn(0x7ff75c0fcb20 msgr2=0x7ff75c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 shutdown_connections 2024-04-24T22:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.183+0000 7ff75be00640 1 -- 172.21.15.111:0/3248182720 wait complete. 2024-04-24T22:41:13.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T22:39:30.113682+0000', 'modified': '2024-04-24T22:39:35.992637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24574, 'mds_2': 24556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/964018737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 964018737}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 964018737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/1876976469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1876976469}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1876976469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/3780798137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3780798137}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3780798137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6839/2246377551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2246377551}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2246377551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24574': {'gid': 24574, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/84522869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 84522869}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 84522869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.171:6839/3821671666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 3821671666}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 3821671666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24587, 'qdb_cluster': [24556, 24574, 24587]} 2024-04-24T22:41:13.206 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T22:41:13.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 -- 172.21.15.111:0/1242882076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54101880 msgr2=0x7f0f54101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 --2- 172.21.15.111:0/1242882076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54101880 0x7f0f54101d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009ca0 tx=0x7f0f4802a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 -- 172.21.15.111:0/1242882076 shutdown_connections 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 --2- 172.21.15.111:0/1242882076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54102240 0x7f0f54108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 --2- 172.21.15.111:0/1242882076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54101880 0x7f0f54101d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 --2- 172.21.15.111:0/1242882076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54100690 0x7f0f54100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 -- 172.21.15.111:0/1242882076 >> 172.21.15.111:0/1242882076 conn(0x7f0f540fbc00 msgr2=0x7f0f540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 -- 172.21.15.111:0/1242882076 shutdown_connections 2024-04-24T22:41:13.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.402+0000 7f0f5c200640 1 -- 172.21.15.111:0/1242882076 wait complete. 2024-04-24T22:41:13.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.403+0000 7f0f5c200640 1 Processor -- start 2024-04-24T22:41:13.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 -- start start 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54100690 0x7f0f54199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54101880 0x7f0f5419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410d490 con 0x7f0f54102240 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410d310 con 0x7f0f54101880 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5c200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410d610 con 0x7f0f54100690 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5a200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5a200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39894/0 (socket says 172.21.15.111:39894) 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f5a200640 1 -- 172.21.15.111:0/4241712136 learned_addr learned my addr 172.21.15.111:0/4241712136 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54101880 0x7f0f5419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.404+0000 7f0f59800640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54100690 0x7f0f54199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 -- 172.21.15.111:0/4241712136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54100690 msgr2=0x7f0f54199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54100690 0x7f0f54199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 -- 172.21.15.111:0/4241712136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54101880 msgr2=0x7f0f5419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54101880 0x7f0f5419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 -- 172.21.15.111:0/4241712136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541a1980 con 0x7f0f54102240 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5a200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00e970 tx=0x7f0f4c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f52000640 1 -- 172.21.15.111:0/4241712136 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00ccc0 con 0x7f0f54102240 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541a1c70 con 0x7f0f54102240 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f52000640 1 -- 172.21.15.111:0/4241712136 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c00ce60 con 0x7f0f54102240 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f52000640 1 -- 172.21.15.111:0/4241712136 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c018630 con 0x7f0f54102240 2024-04-24T22:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.405+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a2190 con 0x7f0f54102240 2024-04-24T22:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 Processor -- start 2024-04-24T22:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f52000640 1 -- 172.21.15.111:0/4241712136 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0f4c0187d0 con 0x7f0f54102240 2024-04-24T22:41:13.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 -- start start 2024-04-24T22:41:13.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbcbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bbd0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bbd3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d3bacbed0 con 0x562d3bbcb760 2024-04-24T22:41:13.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d3bb2edf0 con 0x562d3bbd12c0 2024-04-24T22:41:13.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562d3bad78d0 con 0x562d3bbcc150 2024-04-24T22:41:13.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.410+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbcbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbcbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39900/0 (socket says 172.21.15.111:39900) 2024-04-24T22:41:13.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f58e00640 1 -- 172.21.15.111:0/677690060 learned_addr learned my addr 172.21.15.111:0/677690060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bbd0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f58e00640 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bbd3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 -- 172.21.15.111:0/677690060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 msgr2=0x562d3bbd3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bbd3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 -- 172.21.15.111:0/677690060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 msgr2=0x562d3bbcbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbcbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 -- 172.21.15.111:0/677690060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d3b8432d0 con 0x562d3bbcc150 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.411+0000 7f0f59800640 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bbd0c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009b40 tx=0x7f0f440113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dbfafd7b065c333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f52000640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 0x7f0f3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f42a00640 1 -- 172.21.15.111:0/677690060 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44011e10 con 0x562d3bbcc150 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f42a00640 1 -- 172.21.15.111:0/677690060 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f44012930 con 0x562d3bbcc150 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f42a00640 1 -- 172.21.15.111:0/677690060 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400f9e0 con 0x562d3bbcc150 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f52000640 1 -- 172.21.15.111:0/4241712136 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0f4c0cf600 con 0x7f0f54102240 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f58e00640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 0x7f0f3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f5eff2740 1 -- 172.21.15.111:0/677690060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 msgr2=0x562d3bbd0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f58e00640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 0x7f0f3c0a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562d3bbcc9b0 tx=0x7f0f48009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.412+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bbd0c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009b40 tx=0x7f0f440113e0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 -- 172.21.15.111:0/677690060 shutdown_connections 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bbd3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bbd0c50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/677690060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbcbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 -- 172.21.15.111:0/677690060 >> 172.21.15.111:0/677690060 conn(0x562d3bbc6dd0 msgr2=0x562d3bbc71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 -- 172.21.15.111:0/677690060 shutdown_connections 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 -- 172.21.15.111:0/677690060 wait complete. 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 -- start start 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.413+0000 7f0f5eff2740 1 Processor -- start 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcc150 0x562d3bbd5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbdcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d3baead20 con 0x562d3bbcc150 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d3bb2adb0 con 0x562d3bbcb760 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbd4270 con 0x562d3bbd12c0 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5eff2740 10 client.0 _ll_register_callbacks cb 0x562d3bb25f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcc150 0x562d3bbd5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44280/0 (socket says 172.21.15.111:44280) 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 -- 172.21.15.111:0/765888857 learned_addr learned my addr 172.21.15.111:0/765888857 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f5a200640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbdcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 -- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 msgr2=0x562d3bbdcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbdcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 -- 172.21.15.111:0/765888857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcc150 msgr2=0x562d3bbd5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcc150 0x562d3bbd5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.414+0000 7f0f59800640 1 -- 172.21.15.111:0/765888857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3b90d6b0 con 0x562d3bbcb760 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f59800640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0f4401c040 tx=0x7f0f44012700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8523f9fce554cc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f37400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44005040 con 0x562d3bbcb760 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f44012930 con 0x562d3bbcb760 2024-04-24T22:41:13.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0f440051e0 con 0x562d3bbcb760 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0f44033050 con 0x562d3bbcb760 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f5eff2740 10 client.24980 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.415+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562d3bbe0310 con 0x562d3bbcb760 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f44012930 con 0x562d3bbcb760 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0f440035f0 con 0x562d3bbcb760 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 10 client.24980 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 10 client.24980 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 20 client.24980 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bbe2610 0x562d3bbe3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 4 client.24980 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 --> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d3bb3c330 con 0x562d3bbe2610 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.416+0000 7f0f58e00640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bbe2610 0x562d3bbe3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.417+0000 7f0f58e00640 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bbe2610 0x562d3bbe3a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1360fd633bb37ecb server_cookie=cd5da824ca20ab31 in_seq=0 out_seq=0 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.417+0000 7f0f37400640 10 client.24980 ms_handle_connect on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f37400640 1 -- 172.21.15.111:0/765888857 <== mds.2 v2:172.21.15.186:6838/2246377551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d3bb3c330 con 0x562d3bbe2610 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f37400640 10 client.24980 handle_command_reply: tid=0 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 client.24980 shutdown 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 msgr2=0x562d3bbd59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0f4401c040 tx=0x7f0f44012700 comp rx=0 tx=0).stop 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 shutdown_connections 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bbe2610 0x562d3bbe3a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbdcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcc150 0x562d3bbd5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f37400640 0 client.24980 ms_handle_reset on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/765888857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcb760 0x562d3bbd59d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 >> 172.21.15.111:0/765888857 conn(0x562d3bbc6dd0 msgr2=0x562d3bbc5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 shutdown_connections 2024-04-24T22:41:13.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.418+0000 7f0f5eff2740 1 -- 172.21.15.111:0/765888857 wait complete. 2024-04-24T22:41:13.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.419+0000 7f0f5eff2740 20 client.24980 trim_cache size 0 max 16384 2024-04-24T22:41:13.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 Processor -- start 2024-04-24T22:41:13.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 -- start start 2024-04-24T22:41:13.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbc5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe1dc0 con 0x562d3bbcb760 2024-04-24T22:41:13.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe2490 con 0x562d3bbcc150 2024-04-24T22:41:13.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.438+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe2b60 con 0x562d3bbd12c0 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f59800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44286/0 (socket says 172.21.15.111:44286) 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 -- 172.21.15.111:0/670123346 learned_addr learned my addr 172.21.15.111:0/670123346 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f5a200640 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbc5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 -- 172.21.15.111:0/670123346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 msgr2=0x562d3bbc5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbc5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 -- 172.21.15.111:0/670123346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 msgr2=0x562d3bbc71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbc71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f58e00640 1 -- 172.21.15.111:0/670123346 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d3bbc7b40 con 0x562d3bbcc150 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.439+0000 7f0f5a200640 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbc5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.440+0000 7f0f58e00640 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802aeb0 tx=0x7f0f4803e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20235f3863f3d273 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.440+0000 7f0f42000640 1 -- 172.21.15.111:0/670123346 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4805c070 con 0x562d3bbcc150 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.440+0000 7f0f42000640 1 -- 172.21.15.111:0/670123346 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48036040 con 0x562d3bbcc150 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.440+0000 7f0f42000640 1 -- 172.21.15.111:0/670123346 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802f830 con 0x562d3bbcc150 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- 172.21.15.111:0/670123346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 msgr2=0x562d3bbdcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802aeb0 tx=0x7f0f4803e070 comp rx=0 tx=0).stop 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- 172.21.15.111:0/670123346 shutdown_connections 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbd12c0 0x562d3bbc5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbcc150 0x562d3bbdcf10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/670123346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- 172.21.15.111:0/670123346 >> 172.21.15.111:0/670123346 conn(0x562d3bbc6dd0 msgr2=0x562d3bbc1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- 172.21.15.111:0/670123346 shutdown_connections 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- 172.21.15.111:0/670123346 wait complete. 2024-04-24T22:41:13.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 -- start start 2024-04-24T22:41:13.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.441+0000 7f0f5eff2740 1 Processor -- start 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bc445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bc46af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe3900 con 0x562d3bbcb760 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe3230 con 0x562d3bbd12c0 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562d3bbe3fd0 con 0x562d3bbcc150 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5eff2740 10 client.0 _ll_register_callbacks cb 0x7f0f4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39920/0 (socket says 172.21.15.111:39920) 2024-04-24T22:41:13.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 -- 172.21.15.111:0/2118958889 learned_addr learned my addr 172.21.15.111:0/2118958889 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f58e00640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bc445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5a200640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bc46af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 -- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 msgr2=0x562d3bc445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bc445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 -- 172.21.15.111:0/2118958889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 msgr2=0x562d3bc46af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bc46af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f59800640 1 -- 172.21.15.111:0/2118958889 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3bc47030 con 0x562d3bbcb760 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.442+0000 7f0f5a200640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bc46af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f59800640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009900 tx=0x7f0f44012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e352763ae8d3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f37400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4403b070 con 0x562d3bbcb760 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0f440050f0 con 0x562d3bbcb760 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f48036040 con 0x562d3bbcb760 2024-04-24T22:41:13.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0f44046050 con 0x562d3bbcb760 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f5eff2740 10 client.14889 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.443+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562d3bc48270 con 0x562d3bbcb760 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48036040 con 0x562d3bbcb760 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0f44024a40 con 0x562d3bbcb760 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 10 client.14889 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 10 client.14889 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 20 client.14889 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bc4ce80 0x562d3bc4bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 4 client.14889 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 --> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] -- command(tid 0: {"prefix": "status"}) -- 0x562d3bb36680 con 0x562d3bc4ce80 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.444+0000 7f0f58e00640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bc4ce80 0x562d3bc4bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f58e00640 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bc4ce80 0x562d3bc4bb90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=391863f1fc3277d1 server_cookie=6236dc8376b473da in_seq=0 out_seq=0 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f37400640 10 client.14889 ms_handle_connect on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f37400640 1 -- 172.21.15.111:0/2118958889 <== mds.2 v2:172.21.15.186:6838/2246377551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d3bb36680 con 0x562d3bc4ce80 2024-04-24T22:41:13.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f37400640 10 client.14889 handle_command_reply: tid=0 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 client.14889 shutdown 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 msgr2=0x562d3bbdae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009900 tx=0x7f0f44012a20 comp rx=0 tx=0).stop 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 shutdown_connections 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x562d3bc4ce80 0x562d3bc4bb90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d3bbd12c0 0x562d3bc46af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3bbcc150 0x562d3bc445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 --2- 172.21.15.111:0/2118958889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d3bbcb760 0x562d3bbdae60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f37400640 0 client.14889 ms_handle_reset on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 >> 172.21.15.111:0/2118958889 conn(0x562d3bbc6dd0 msgr2=0x562d3bbc71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 shutdown_connections 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 1 -- 172.21.15.111:0/2118958889 wait complete. 2024-04-24T22:41:13.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.445+0000 7f0f5eff2740 20 client.14889 trim_cache size 0 max 16384 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 2, 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24556, 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 98.469184842999994, 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:13.454 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:13.455 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:13.455 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 118.734808902 2024-04-24T22:41:13.455 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:13.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 msgr2=0x7f0f3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 0x7f0f3c0a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562d3bbcc9b0 tx=0x7f0f48009360 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 msgr2=0x7f0f541a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00e970 tx=0x7f0f4c00ed70 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 shutdown_connections 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0f3c0a2b90 0x7f0f3c0a5050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f54102240 0x7f0f541a1310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f54101880 0x7f0f5419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 --2- 172.21.15.111:0/4241712136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f54100690 0x7f0f54199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 >> 172.21.15.111:0/4241712136 conn(0x7f0f540fbc00 msgr2=0x7f0f54109490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 shutdown_connections 2024-04-24T22:41:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.446+0000 7f0f5c200640 1 -- 172.21.15.111:0/4241712136 wait complete. 2024-04-24T22:41:13.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-24T22:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 -- 172.21.15.111:0/2696183824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 msgr2=0x7f8248073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 --2- 172.21.15.111:0/2696183824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f8248073bd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f823c009930 tx=0x7f823c02a420 comp rx=0 tx=0).stop 2024-04-24T22:41:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 -- 172.21.15.111:0/2696183824 shutdown_connections 2024-04-24T22:41:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 --2- 172.21.15.111:0/2696183824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f8248110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 --2- 172.21.15.111:0/2696183824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 0x7f8248074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 --2- 172.21.15.111:0/2696183824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f8248073bd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 -- 172.21.15.111:0/2696183824 >> 172.21.15.111:0/2696183824 conn(0x7f82480fcbe0 msgr2=0x7f82480ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 -- 172.21.15.111:0/2696183824 shutdown_connections 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.654+0000 7f824de00640 1 -- 172.21.15.111:0/2696183824 wait complete. 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 Processor -- start 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 -- start start 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 0x7f824819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8248113910 con 0x7f8248075780 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8248113790 con 0x7f82480741a0 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.655+0000 7f824de00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8248113a90 con 0x7f824810dc20 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f8247e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52014/0 (socket says 172.21.15.111:52014) 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f8247e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39930/0 (socket says 172.21.15.111:39930) 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 -- 172.21.15.111:0/4264288751 learned_addr learned my addr 172.21.15.111:0/4264288751 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824ca00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 0x7f824819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 -- 172.21.15.111:0/4264288751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 msgr2=0x7f824819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 0x7f824819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 -- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 msgr2=0x7f824819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 -- 172.21.15.111:0/4264288751 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a5ba0 con 0x7f824810dc20 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f8247e00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.656+0000 7f824d400640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f824000eb10 tx=0x7f824000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.657+0000 7f8245600640 1 -- 172.21.15.111:0/4264288751 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240004b20 con 0x7f824810dc20 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.657+0000 7f8245600640 1 -- 172.21.15.111:0/4264288751 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240004cc0 con 0x7f824810dc20 2024-04-24T22:41:13.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.657+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a5e90 con 0x7f824810dc20 2024-04-24T22:41:13.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.657+0000 7f8245600640 1 -- 172.21.15.111:0/4264288751 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240010760 con 0x7f824810dc20 2024-04-24T22:41:13.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.657+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a63a0 con 0x7f824810dc20 2024-04-24T22:41:13.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.660+0000 7f82521f6740 1 Processor -- start 2024-04-24T22:41:13.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.660+0000 7f8245600640 1 -- 172.21.15.111:0/4264288751 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8240005020 con 0x7f824810dc20 2024-04-24T22:41:13.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 -- start start 2024-04-24T22:41:13.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 0x55fc1a6d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a5d1ed0 con 0x55fc1a6d1970 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a634df0 con 0x55fc1a6d2360 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f82521f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a5dd8d0 con 0x55fc1a6d74d0 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f8247e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 0x55fc1a6d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44324/0 (socket says 172.21.15.111:44324) 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 -- 172.21.15.111:0/3054788231 learned_addr learned my addr 172.21.15.111:0/3054788231 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f8247e00640 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 -- 172.21.15.111:0/3054788231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 msgr2=0x55fc1a6d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 -- 172.21.15.111:0/3054788231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 msgr2=0x55fc1a6d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.661+0000 7f824ca00640 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 0x55fc1a6d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f824ca00640 1 -- 172.21.15.111:0/3054788231 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1a3492d0 con 0x55fc1a6d2360 2024-04-24T22:41:13.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8247e00640 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 0x55fc1a6d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f824ca00640 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f823c031440 tx=0x7f823c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6134e166a8a24715 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8245600640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 0x7f82300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8239c00640 1 -- 172.21.15.111:0/3054788231 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c040070 con 0x55fc1a6d2360 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8239c00640 1 -- 172.21.15.111:0/3054788231 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c004400 con 0x55fc1a6d2360 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8239c00640 1 -- 172.21.15.111:0/3054788231 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c035040 con 0x55fc1a6d2360 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.662+0000 7f8245600640 1 -- 172.21.15.111:0/4264288751 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f82400cf6b0 con 0x7f824810dc20 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f8247e00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 0x7f82300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 -- 172.21.15.111:0/3054788231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 msgr2=0x55fc1a6d6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f823c031440 tx=0x7f823c005df0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 -- 172.21.15.111:0/3054788231 shutdown_connections 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6d6e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 --2- 172.21.15.111:0/3054788231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d1970 0x55fc1a6d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.663+0000 7f82521f6740 1 -- 172.21.15.111:0/3054788231 >> 172.21.15.111:0/3054788231 conn(0x55fc1a6ccfe0 msgr2=0x55fc1a6cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.664+0000 7f8247e00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 0x7f82300a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55fc1a6d2bc0 tx=0x7f8234015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.664+0000 7f82521f6740 1 -- 172.21.15.111:0/3054788231 shutdown_connections 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.664+0000 7f82521f6740 1 -- 172.21.15.111:0/3054788231 wait complete. 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.664+0000 7f82521f6740 1 -- start start 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.664+0000 7f82521f6740 1 Processor -- start 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d2360 0x55fc1a6dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a5f0d20 con 0x55fc1a6d74d0 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a630db0 con 0x55fc1a6d1970 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6da420 con 0x55fc1a6d2360 2024-04-24T22:41:13.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f82521f6740 10 client.0 _ll_register_callbacks cb 0x55fc1a62bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44334/0 (socket says 172.21.15.111:44334) 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f824ca00640 1 -- 172.21.15.111:0/2648090727 learned_addr learned my addr 172.21.15.111:0/2648090727 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f8247e00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d2360 0x55fc1a6dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.665+0000 7f824d400640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 -- 172.21.15.111:0/2648090727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d2360 msgr2=0x55fc1a6dc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d2360 0x55fc1a6dc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 -- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 msgr2=0x55fc1a6e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 -- 172.21.15.111:0/2648090727 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1a4136b0 con 0x55fc1a6d1970 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824d400640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f824ca00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f823c006010 tx=0x7f823c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d30249f12a831b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f822e000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:13.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c035070 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f823c005000 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.666+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823c004400 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f823c040040 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f82521f6740 10 client.24998 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fc1a6e6510 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c004400 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f823c0035f0 con 0x55fc1a6d1970 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f82521f6740 10 client.24998 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.667+0000 7f82521f6740 10 client.24998 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f82521f6740 20 client.24998 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a6e8730 0x55fc1a6e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f82521f6740 4 client.24998 mds_command: new command op to 24574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 --> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1a642330 con 0x55fc1a6e8730 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f8247e00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a6e8730 0x55fc1a6e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f8247e00640 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a6e8730 0x55fc1a6e9ba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b67187c5d54e5991 server_cookie=85053fe6a339c01e in_seq=0 out_seq=0 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.668+0000 7f822e000640 10 client.24998 ms_handle_connect on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.669+0000 7f822e000640 1 -- 172.21.15.111:0/2648090727 <== mds.1 v2:172.21.15.111:6838/84522869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc1a642330 con 0x55fc1a6e8730 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.669+0000 7f822e000640 10 client.24998 handle_command_reply: tid=0 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.669+0000 7f82521f6740 1 client.24998 shutdown 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 msgr2=0x55fc1a6dbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f823c006010 tx=0x7f823c009680 comp rx=0 tx=0).stop 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 shutdown_connections 2024-04-24T22:41:13.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a6e8730 0x55fc1a6e9ba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d2360 0x55fc1a6dc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 --2- 172.21.15.111:0/2648090727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6dbc20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 >> 172.21.15.111:0/2648090727 conn(0x55fc1a6ccfe0 msgr2=0x55fc1a6cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f822e000640 0 client.24998 ms_handle_reset on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 shutdown_connections 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 1 -- 172.21.15.111:0/2648090727 wait complete. 2024-04-24T22:41:13.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.670+0000 7f82521f6740 20 client.24998 trim_cache size 0 max 16384 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 Processor -- start 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 -- start start 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d1970 0x55fc1a6c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6e7ee0 con 0x55fc1a6d74d0 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6e85b0 con 0x55fc1a6d2360 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.690+0000 7f82521f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6e8c80 con 0x55fc1a6d1970 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f824d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f824d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39946/0 (socket says 172.21.15.111:39946) 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44348/0 (socket says 172.21.15.111:44348) 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f824d400640 1 -- 172.21.15.111:0/4148234457 learned_addr learned my addr 172.21.15.111:0/4148234457 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f824ca00640 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d1970 0x55fc1a6c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 -- 172.21.15.111:0/4148234457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d1970 msgr2=0x55fc1a6c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d1970 0x55fc1a6c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 -- 172.21.15.111:0/4148234457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 msgr2=0x55fc1a6cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f8247e00640 1 -- 172.21.15.111:0/4148234457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1a6cdd50 con 0x55fc1a6d2360 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.691+0000 7f824d400640 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.692+0000 7f8247e00640 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55fc1a6dd100 tx=0x7f82340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f79100e0c4c7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.692+0000 7f8239200640 1 -- 172.21.15.111:0/4148234457 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234019030 con 0x55fc1a6d2360 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.692+0000 7f8239200640 1 -- 172.21.15.111:0/4148234457 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823401b050 con 0x55fc1a6d2360 2024-04-24T22:41:13.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.692+0000 7f8239200640 1 -- 172.21.15.111:0/4148234457 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234021890 con 0x55fc1a6d2360 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- 172.21.15.111:0/4148234457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 msgr2=0x55fc1a6e3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55fc1a6dd100 tx=0x7f82340046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- 172.21.15.111:0/4148234457 shutdown_connections 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d74d0 0x55fc1a6cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d2360 0x55fc1a6e3160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 --2- 172.21.15.111:0/4148234457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d1970 0x55fc1a6c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- 172.21.15.111:0/4148234457 >> 172.21.15.111:0/4148234457 conn(0x55fc1a6ccfe0 msgr2=0x55fc1a6d2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- 172.21.15.111:0/4148234457 shutdown_connections 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- 172.21.15.111:0/4148234457 wait complete. 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 -- start start 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.693+0000 7f82521f6740 1 Processor -- start 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 0x55fc1a74a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a74cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6e9a20 con 0x55fc1a6d2360 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6e9350 con 0x55fc1a6d1970 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a6ea0f0 con 0x55fc1a6d74d0 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f82521f6740 10 client.0 _ll_register_callbacks cb 0x7f823c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f8247e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 0x55fc1a74a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f824ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44354/0 (socket says 172.21.15.111:44354) 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f824ca00640 1 -- 172.21.15.111:0/2480789746 learned_addr learned my addr 172.21.15.111:0/2480789746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.694+0000 7f824ca00640 1 -- 172.21.15.111:0/2480789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 msgr2=0x55fc1a74cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f824ca00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a74cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f824ca00640 1 -- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 msgr2=0x55fc1a74a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f824ca00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 0x55fc1a74a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f824ca00640 1 -- 172.21.15.111:0/2480789746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1a74d2d0 con 0x55fc1a6d1970 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f8247e00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 0x55fc1a74a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f824ca00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f823c0058b0 tx=0x7f823c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27561a516e5bfbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f822e000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c035040 con 0x55fc1a6d1970 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f823c004410 con 0x55fc1a6d1970 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823401b050 con 0x55fc1a6d1970 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f823c003be0 con 0x55fc1a6d1970 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f82521f6740 10 client.25004 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.695+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fc1a74e5d0 con 0x55fc1a6d1970 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823401b050 con 0x55fc1a6d1970 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f823c0316b0 con 0x55fc1a6d1970 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 10 client.25004 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 10 client.25004 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 20 client.25004 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a753220 0x55fc1a751f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 4 client.25004 mds_command: new command op to 24574 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 --> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] -- command(tid 0: {"prefix": "status"}) -- 0x55fc1a63c680 con 0x55fc1a753220 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.696+0000 7f8247e00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a753220 0x55fc1a751f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f822e000640 10 client.25004 ms_handle_connect on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f8247e00640 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a753220 0x55fc1a751f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b90efcb229a72e1d server_cookie=1d2d366b11eb1d28 in_seq=0 out_seq=0 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f822e000640 1 -- 172.21.15.111:0/2480789746 <== mds.1 v2:172.21.15.111:6838/84522869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55fc1a63c680 con 0x55fc1a753220 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f822e000640 10 client.25004 handle_command_reply: tid=0 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f82521f6740 1 client.25004 shutdown 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 msgr2=0x55fc1a6e1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.697+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f823c0058b0 tx=0x7f823c004a30 comp rx=0 tx=0).stop 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 shutdown_connections 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fc1a6d74d0 0x55fc1a74cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x55fc1a753220 0x55fc1a751f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a6d2360 0x55fc1a74a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 --2- 172.21.15.111:0/2480789746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fc1a6d1970 0x55fc1a6e1150 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f822e000640 0 client.25004 ms_handle_reset on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 >> 172.21.15.111:0/2480789746 conn(0x55fc1a6ccfe0 msgr2=0x55fc1a6cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 shutdown_connections 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 1 -- 172.21.15.111:0/2480789746 wait complete. 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.698+0000 7f82521f6740 20 client.25004 trim_cache size 0 max 16384 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 1, 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24574, 2024-04-24T22:41:13.706 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 99.735811431000002, 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 115.01822188 2024-04-24T22:41:13.707 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 msgr2=0x7f82300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 0x7f82300a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55fc1a6d2bc0 tx=0x7f8234015040 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 msgr2=0x7f82481a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f824000eb10 tx=0x7f824000c2d0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 shutdown_connections 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f82300a2c10 0x7f82300a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810dc20 0x7f82481a5530 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8248075780 0x7f824819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 --2- 172.21.15.111:0/4264288751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82480741a0 0x7f824819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 >> 172.21.15.111:0/4264288751 conn(0x7f82480fcbe0 msgr2=0x7f8248100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 shutdown_connections 2024-04-24T22:41:13.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.699+0000 7f824de00640 1 -- 172.21.15.111:0/4264288751 wait complete. 2024-04-24T22:41:13.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 -- 172.21.15.111:0/1887732253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c105430 msgr2=0x7f4e6c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 --2- 172.21.15.111:0/1887732253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c105430 0x7f4e6c10bcc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ca0 tx=0x7f4e6402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 -- 172.21.15.111:0/1887732253 shutdown_connections 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 --2- 172.21.15.111:0/1887732253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c105430 0x7f4e6c10bcc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 --2- 172.21.15.111:0/1887732253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 0x7f4e6c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 --2- 172.21.15.111:0/1887732253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c103880 0x7f4e6c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 -- 172.21.15.111:0/1887732253 >> 172.21.15.111:0/1887732253 conn(0x7f4e6c0fedf0 msgr2=0x7f4e6c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 -- 172.21.15.111:0/1887732253 shutdown_connections 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.916+0000 7f4e71e00640 1 -- 172.21.15.111:0/1887732253 wait complete. 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.917+0000 7f4e71e00640 1 Processor -- start 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.917+0000 7f4e71e00640 1 -- start start 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c103880 0x7f4e6c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 0x7f4e6c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c10f550 con 0x7f4e6c105430 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c10f3d0 con 0x7f4e6c104a70 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c10f6d0 con 0x7f4e6c103880 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c103880 0x7f4e6c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39980/0 (socket says 172.21.15.111:39980) 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 0x7f4e6c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71400640 1 -- 172.21.15.111:0/3725476840 learned_addr learned my addr 172.21.15.111:0/3725476840 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71400640 1 -- 172.21.15.111:0/3725476840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c103880 msgr2=0x7f4e6c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.918+0000 7f4e71400640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c103880 0x7f4e6c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71400640 1 -- 172.21.15.111:0/3725476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 msgr2=0x7f4e6c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71400640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 0x7f4e6c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71400640 1 -- 172.21.15.111:0/3725476840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c1a1700 con 0x7f4e6c105430 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71400640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e6402f540 tx=0x7f4e64031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e6a000640 1 -- 172.21.15.111:0/3725476840 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64003b70 con 0x7f4e6c105430 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c1a1990 con 0x7f4e6c105430 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e6a000640 1 -- 172.21.15.111:0/3725476840 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e64003d10 con 0x7f4e6c105430 2024-04-24T22:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c1a1f30 con 0x7f4e6c105430 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.919+0000 7f4e6a000640 1 -- 172.21.15.111:0/3725476840 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402fb10 con 0x7f4e6c105430 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.922+0000 7f4e761ce740 1 Processor -- start 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.922+0000 7f4e6a000640 1 -- 172.21.15.111:0/3725476840 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4e64004640 con 0x7f4e6c105430 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 -- start start 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02afbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02afe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c029f6ed0 con 0x560c02afc4d0 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c02a59df0 con 0x560c02af6970 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e761ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c02a028d0 con 0x560c02af7360 2024-04-24T22:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02afe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02afbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44374/0 (socket says 172.21.15.111:44374) 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.923+0000 7f4e63e00640 1 -- 172.21.15.111:0/785306884 learned_addr learned my addr 172.21.15.111:0/785306884 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 -- 172.21.15.111:0/785306884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 msgr2=0x560c02af6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 -- 172.21.15.111:0/785306884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 msgr2=0x560c02afe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02afe960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 -- 172.21.15.111:0/785306884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0276e2d0 con 0x560c02af7360 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e63e00640 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02afe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e6a000640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 0x7f4e500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e6a000640 1 -- 172.21.15.111:0/3725476840 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4e640f0570 con 0x7f4e6c105430 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e70a00640 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02afbe60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009bf0 tx=0x7f4e5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbef5d8d7013d514 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.924+0000 7f4e63e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 0x7f4e500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e61c00640 1 -- 172.21.15.111:0/785306884 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c009da0 con 0x560c02af7360 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e61c00640 1 -- 172.21.15.111:0/785306884 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c005020 con 0x560c02af7360 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e61c00640 1 -- 172.21.15.111:0/785306884 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004b60 con 0x560c02af7360 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e761ce740 1 -- 172.21.15.111:0/785306884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 msgr2=0x560c02afbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e63e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 0x7f4e500a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x560c02af7bc0 tx=0x7f4e54004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.925+0000 7f4e761ce740 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02afbe60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009bf0 tx=0x7f4e5c011810 comp rx=0 tx=0).stop 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 -- 172.21.15.111:0/785306884 shutdown_connections 2024-04-24T22:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02afe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02afbe60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 --2- 172.21.15.111:0/785306884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 -- 172.21.15.111:0/785306884 >> 172.21.15.111:0/785306884 conn(0x560c02af1fe0 msgr2=0x560c02af23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 -- 172.21.15.111:0/785306884 shutdown_connections 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 -- 172.21.15.111:0/785306884 wait complete. 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 -- start start 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.926+0000 7f4e761ce740 1 Processor -- start 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 0x560c02b011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02afc4d0 0x560c02b081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c02a15d20 con 0x560c02af7360 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c02a55db0 con 0x560c02af6970 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c02aff480 con 0x560c02afc4d0 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e761ce740 10 client.0 _ll_register_callbacks cb 0x560c02a50f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 0x560c02b011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:44388/0 (socket says 172.21.15.111:44388) 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 -- 172.21.15.111:0/3330198161 learned_addr learned my addr 172.21.15.111:0/3330198161 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e71400640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02afc4d0 0x560c02b081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 -- 172.21.15.111:0/3330198161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02afc4d0 msgr2=0x560c02b081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02afc4d0 0x560c02b081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 -- 172.21.15.111:0/3330198161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 msgr2=0x560c02b011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 0x560c02b011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e70a00640 1 -- 172.21.15.111:0/3330198161 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c028386b0 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.927+0000 7f4e63e00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 0x560c02b011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e70a00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c01b040 tx=0x7f4e5c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc623b0958d050e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e5a000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c029020 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c005020 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4e5c004ab0 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c004cb0 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e761ce740 10 client.25016 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.928+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560c02b0b5c0 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c005020 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4e5c019450 con 0x560c02af6970 2024-04-24T22:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 10 client.25016 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 10 client.25016 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 20 client.25016 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b0d9a0 0x560c02b0ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 4 client.25016 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c02a67330 con 0x560c02b0d9a0 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.929+0000 7f4e63e00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b0d9a0 0x560c02b0ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.930+0000 7f4e63e00640 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b0d9a0 0x560c02b0ee10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc54b9d10b9d438 server_cookie=3bcb6dd7e2040912 in_seq=0 out_seq=0 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.930+0000 7f4e5a000640 10 client.25016 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e5a000640 1 -- 172.21.15.111:0/3330198161 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c02a67330 con 0x560c02b0d9a0 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e5a000640 10 client.25016 handle_command_reply: tid=0 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 client.25016 shutdown 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 msgr2=0x560c02b00ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c01b040 tx=0x7f4e5c009fd0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 shutdown_connections 2024-04-24T22:41:13.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b0d9a0 0x560c02b0ee10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02afc4d0 0x560c02b081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af7360 0x560c02b011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 --2- 172.21.15.111:0/3330198161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02b00ca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e5a000640 0 client.25016 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 >> 172.21.15.111:0/3330198161 conn(0x560c02af1fe0 msgr2=0x560c02af1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.931+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 shutdown_connections 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.932+0000 7f4e761ce740 1 -- 172.21.15.111:0/3330198161 wait complete. 2024-04-24T22:41:13.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.932+0000 7f4e761ce740 20 client.25016 trim_cache size 0 max 16384 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 Processor -- start 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 -- start start 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0d150 con 0x560c02afc4d0 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0d820 con 0x560c02af6970 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.951+0000 7f4e761ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0def0 con 0x560c02af7360 2024-04-24T22:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40014/0 (socket says 172.21.15.111:40014) 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 -- 172.21.15.111:0/319752103 learned_addr learned my addr 172.21.15.111:0/319752103 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e70a00640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e63e00640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 -- 172.21.15.111:0/319752103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 msgr2=0x560c02b081a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b081a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 -- 172.21.15.111:0/319752103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 msgr2=0x560c02af0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 -- 172.21.15.111:0/319752103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c02b0f140 con 0x560c02afc4d0 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e70a00640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e63e00640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.952+0000 7f4e71400640 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e640ee020 tx=0x7f4e6402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341936eff68bd44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e61200640 1 -- 172.21.15.111:0/319752103 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640bcd80 con 0x560c02afc4d0 2024-04-24T22:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e61200640 1 -- 172.21.15.111:0/319752103 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e64036040 con 0x560c02afc4d0 2024-04-24T22:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e61200640 1 -- 172.21.15.111:0/319752103 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640b9560 con 0x560c02afc4d0 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 -- 172.21.15.111:0/319752103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 msgr2=0x560c02af0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e640ee020 tx=0x7f4e6402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 -- 172.21.15.111:0/319752103 shutdown_connections 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02afc4d0 0x560c02af0dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 --2- 172.21.15.111:0/319752103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02af6970 0x560c02af0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 -- 172.21.15.111:0/319752103 >> 172.21.15.111:0/319752103 conn(0x560c02af1fe0 msgr2=0x560c02af23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 -- 172.21.15.111:0/319752103 shutdown_connections 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.953+0000 7f4e761ce740 1 -- 172.21.15.111:0/319752103 wait complete. 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 -- start start 2024-04-24T22:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 Processor -- start 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02afc4d0 0x560c02b07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0ec90 con 0x560c02af6970 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0e5c0 con 0x560c02afc4d0 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c02b0f360 con 0x560c02af7360 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.954+0000 7f4e761ce740 10 client.0 _ll_register_callbacks cb 0x7f4e54009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40020/0 (socket says 172.21.15.111:40020) 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 -- 172.21.15.111:0/1387418963 learned_addr learned my addr 172.21.15.111:0/1387418963 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e63e00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e71400640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02afc4d0 0x560c02b07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 -- 172.21.15.111:0/1387418963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 msgr2=0x560c02b040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 -- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02afc4d0 msgr2=0x560c02b07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02afc4d0 0x560c02b07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 -- 172.21.15.111:0/1387418963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c02b076f0 con 0x560c02af6970 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e70a00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c019a10 tx=0x7f4e5c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d937da055eb94564 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.955+0000 7f4e5a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c029040 con 0x560c02af6970 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4e5c019a90 con 0x560c02af6970 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e64036040 con 0x560c02af6970 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c019cb0 con 0x560c02af6970 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 10 client.14925 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560c02b7c390 con 0x560c02af6970 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e64036040 con 0x560c02af6970 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4e5c037020 con 0x560c02af6970 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 10 client.14925 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 10 client.14925 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 20 client.14925 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b80ff0 0x560c02b7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 4 client.14925 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.956+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "status"}) -- 0x560c02a61680 con 0x560c02b80ff0 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.957+0000 7f4e63e00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b80ff0 0x560c02b7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e63e00640 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b80ff0 0x560c02b7d430 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7bf1e7269edc5f server_cookie=308b48e84e92c1cd in_seq=0 out_seq=0 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e5a000640 10 client.14925 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e5a000640 1 -- 172.21.15.111:0/1387418963 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560c02a61680 con 0x560c02b80ff0 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e5a000640 10 client.14925 handle_command_reply: tid=0 2024-04-24T22:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 client.14925 shutdown 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 msgr2=0x560c02b03ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c019a10 tx=0x7f4e5c009310 comp rx=0 tx=0).stop 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 shutdown_connections 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c02b80ff0 0x560c02b7d430 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c02afc4d0 0x560c02b07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c02af7360 0x560c02b040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e5a000640 0 client.14925 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 --2- 172.21.15.111:0/1387418963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c02af6970 0x560c02b03ba0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 >> 172.21.15.111:0/1387418963 conn(0x560c02af1fe0 msgr2=0x560c02af4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 shutdown_connections 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 1 -- 172.21.15.111:0/1387418963 wait complete. 2024-04-24T22:41:13.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.958+0000 7f4e761ce740 20 client.14925 trim_cache size 0 max 16384 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24587, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 103.28897340899999, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 108.493345894 2024-04-24T22:41:13.968 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 msgr2=0x7f4e500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 0x7f4e500a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x560c02af7bc0 tx=0x7f4e54004730 comp rx=0 tx=0).stop 2024-04-24T22:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 msgr2=0x7f4e6c1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e6402f540 tx=0x7f4e64031750 comp rx=0 tx=0).stop 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 shutdown_connections 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4e500a2b90 0x7f4e500a5050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e6c105430 0x7f4e6c1a1090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e6c104a70 0x7f4e6c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 --2- 172.21.15.111:0/3725476840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c103880 0x7f4e6c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 >> 172.21.15.111:0/3725476840 conn(0x7f4e6c0fedf0 msgr2=0x7f4e6c107ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 shutdown_connections 2024-04-24T22:41:13.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:13.959+0000 7f4e71e00640 1 -- 172.21.15.111:0/3725476840 wait complete. 2024-04-24T22:41:13.977 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:13 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3248182720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:13.979 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T22:41:13.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:41:13.995 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:13.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T22:41:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:13 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3248182720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:14.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:13 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3248182720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:14.982 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:14 smithi111 ceph-mon[27746]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-24T22:41:15.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:14 smithi171 ceph-mon[32337]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-24T22:41:15.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:14 smithi186 ceph-mon[32753]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-24T22:41:16.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:16 smithi111 ceph-mon[27746]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:16.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 -- 172.21.15.111:0/600164808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b5010a170 msgr2=0x7f9b5010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 --2- 172.21.15.111:0/600164808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b5010a170 0x7f9b5010c600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009990 tx=0x7f9b48009ca0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 -- 172.21.15.111:0/600164808 shutdown_connections 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 --2- 172.21.15.111:0/600164808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b5010a170 0x7f9b5010c600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 --2- 172.21.15.111:0/600164808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b50076260 0x7f9b500746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 --2- 172.21.15.111:0/600164808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b50075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 -- 172.21.15.111:0/600164808 >> 172.21.15.111:0/600164808 conn(0x7f9b500fd800 msgr2=0x7f9b500ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 -- 172.21.15.111:0/600164808 shutdown_connections 2024-04-24T22:41:16.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.798+0000 7f9b56400700 1 -- 172.21.15.111:0/600164808 wait complete. 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.799+0000 7f9b56400700 1 Processor -- start 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.799+0000 7f9b56400700 1 -- start start 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b5019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b5010a170 0x7f9b501a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501100e0 con 0x7f9b50076260 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010ff60 con 0x7f9b5010a170 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b56400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50110260 con 0x7f9b50074c90 2024-04-24T22:41:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b47e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b47e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41098/0 (socket says 172.21.15.111:41098) 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b4f400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b5019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b47e00700 1 -- 172.21.15.111:0/635850726 learned_addr learned my addr 172.21.15.111:0/635850726 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.800+0000 7f9b4fe00700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b5010a170 0x7f9b501a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 -- 172.21.15.111:0/635850726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b5010a170 msgr2=0x7f9b501a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b5010a170 0x7f9b501a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 -- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 msgr2=0x7f9b5019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 -- 172.21.15.111:0/635850726 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b501a22f0 con 0x7f9b50074c90 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b47e00700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4f400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b5019a690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000efc0 tx=0x7f9b4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400092e0 con 0x7f9b50074c90 2024-04-24T22:41:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4000ca20 con 0x7f9b50074c90 2024-04-24T22:41:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b501a2580 con 0x7f9b50074c90 2024-04-24T22:41:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400045c0 con 0x7f9b50074c90 2024-04-24T22:41:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.801+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a2ac0 con 0x7f9b50074c90 2024-04-24T22:41:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.802+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50068f20 con 0x7f9b50074c90 2024-04-24T22:41:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.804+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9b4001b030 con 0x7f9b50074c90 2024-04-24T22:41:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.806+0000 7f9b4cc00700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 0x7f9b340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.806+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9b400ce860 con 0x7f9b50074c90 2024-04-24T22:41:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.806+0000 7f9b47e00700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 0x7f9b340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.807+0000 7f9b47e00700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 0x7f9b340a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005950 tx=0x7f9b38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:16.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:16.810+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b40022060 con 0x7f9b50074c90 2024-04-24T22:41:17.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.054+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9b50062670 con 0x7f9b340a2c20 2024-04-24T22:41:17.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.057+0000 7f9b4cc00700 1 -- 172.21.15.111:0/635850726 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9b50062670 con 0x7f9b340a2c20 2024-04-24T22:41:17.060 INFO:teuthology.orchestra.run.smithi111.stdout:Backend: cephadm 2024-04-24T22:41:17.061 INFO:teuthology.orchestra.run.smithi111.stdout:Available: Yes 2024-04-24T22:41:17.061 INFO:teuthology.orchestra.run.smithi111.stdout:Paused: No 2024-04-24T22:41:17.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 msgr2=0x7f9b340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 0x7f9b340a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005950 tx=0x7f9b38009260 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 msgr2=0x7f9b5019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b5019a690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000efc0 tx=0x7f9b4000c3a0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 shutdown_connections 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9b340a2c20 0x7f9b340a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b5010a170 0x7f9b501a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b50074c90 0x7f9b5019a690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 --2- 172.21.15.111:0/635850726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b50076260 0x7f9b5019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.061+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 >> 172.21.15.111:0/635850726 conn(0x7f9b500fd800 msgr2=0x7f9b50106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.062+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 shutdown_connections 2024-04-24T22:41:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:17.062+0000 7f9b56400700 1 -- 172.21.15.111:0/635850726 wait complete. 2024-04-24T22:41:17.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:16 smithi171 ceph-mon[32337]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:16 smithi186 ceph-mon[32753]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:17.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T22:41:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:19.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:18 smithi111 ceph-mon[27746]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:19.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:18 smithi111 ceph-mon[27746]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:19.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:18 smithi171 ceph-mon[32337]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:19.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:18 smithi171 ceph-mon[32337]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:18 smithi186 ceph-mon[32753]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:18 smithi186 ceph-mon[32753]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 -- 172.21.15.111:0/52344208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103e30 msgr2=0x7f041410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 --2- 172.21.15.111:0/52344208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103e30 0x7f041410a6e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f040c00b0d0 tx=0x7f040c00b3e0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 -- 172.21.15.111:0/52344208 shutdown_connections 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 --2- 172.21.15.111:0/52344208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103e30 0x7f041410a6e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 --2- 172.21.15.111:0/52344208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414102290 0x7f04141026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 --2- 172.21.15.111:0/52344208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414103490 0x7f04141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 -- 172.21.15.111:0/52344208 >> 172.21.15.111:0/52344208 conn(0x7f04140fd830 msgr2=0x7f04140ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 -- 172.21.15.111:0/52344208 shutdown_connections 2024-04-24T22:41:20.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.621+0000 7f041a800700 1 -- 172.21.15.111:0/52344208 wait complete. 2024-04-24T22:41:20.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.622+0000 7f041a800700 1 Processor -- start 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 -- start start 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414102290 0x7f041419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414103e30 0x7f04141a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f041410de70 con 0x7f0414103490 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f041410dcf0 con 0x7f0414102290 2024-04-24T22:41:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f041a800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f041410dff0 con 0x7f0414103e30 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f0412a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f0412a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41110/0 (socket says 172.21.15.111:41110) 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f0412a00700 1 -- 172.21.15.111:0/2900039099 learned_addr learned my addr 172.21.15.111:0/2900039099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.623+0000 7f0413400700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414102290 0x7f041419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0413e00700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414103e30 0x7f04141a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 -- 172.21.15.111:0/2900039099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414103e30 msgr2=0x7f04141a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414103e30 0x7f04141a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 -- 172.21.15.111:0/2900039099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414102290 msgr2=0x7f041419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414102290 0x7f041419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 -- 172.21.15.111:0/2900039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04141a22b0 con 0x7f0414103490 2024-04-24T22:41:20.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.624+0000 7f0412a00700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f040000e9d0 tx=0x7f040000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.625+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040000cbd0 con 0x7f0414103490 2024-04-24T22:41:20.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.625+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04141a25a0 con 0x7f0414103490 2024-04-24T22:41:20.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.625+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040000ee70 con 0x7f0414103490 2024-04-24T22:41:20.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.625+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400018680 con 0x7f0414103490 2024-04-24T22:41:20.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.625+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a2b30 con 0x7f0414103490 2024-04-24T22:41:20.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.626+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0414068f20 con 0x7f0414103490 2024-04-24T22:41:20.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.629+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f04000188e0 con 0x7f0414103490 2024-04-24T22:41:20.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.630+0000 7f040be00700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 0x7f03f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.630+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f04000cf290 con 0x7f0414103490 2024-04-24T22:41:20.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.630+0000 7f0413400700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 0x7f03f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.631+0000 7f0413400700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 0x7f03f80a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f04141032f0 tx=0x7f040400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.639+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f040009c1a0 con 0x7f0414103490 2024-04-24T22:41:20.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.878+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0414075b80 con 0x7f03f80a2c20 2024-04-24T22:41:20.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:20 smithi111 ceph-mon[27746]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:20.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.915+0000 7f040be00700 1 -- 172.21.15.111:0/2900039099 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f0414075b80 con 0x7f03f80a2c20 2024-04-24T22:41:20.918 INFO:teuthology.orchestra.run.smithi111.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.a smithi111 running (118s) 22s ago 118s 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef d19031486cb8 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.b smithi171 running (115s) 47s ago 115s 25.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 07004eddc28a 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.c smithi186 running (2m) 47s ago 2m 15.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0a5408d35551 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.d smithi111 running (2m) 22s ago 2m 12.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea30de002ade 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.e smithi171 running (2m) 47s ago 2m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef b24210f614ad 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.f smithi186 running (2m) 47s ago 2m 19.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 43e7a9537ce5 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.g smithi111 running (2m) 22s ago 2m 16.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 533bcc9d208d 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.h smithi171 running (2m) 47s ago 2m 20.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 707acaf489e4 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.i smithi186 running (2m) 47s ago 2m 16.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef 57e6ca4126c8 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.j smithi111 running (2m) 22s ago 2m 12.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 6bcaa13fd6c2 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.k smithi171 running (2m) 47s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 61cee8c93434 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mds.l smithi186 running (2m) 47s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 39fcf77a2abb 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.x smithi111 *:9283,8765 running (10m) 22s ago 10m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef a50fb1ee3480 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.y smithi171 *:8443,8765 running (8m) 47s ago 8m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8df2310a3196 2024-04-24T22:41:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.z smithi186 *:8443,8765 running (8m) 47s ago 8m 454M - 19.0.0-2445-gd2c45d97 f68e37b807ef 203973cd9bea 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:mon.a smithi111 running (10m) 22s ago 11m 48.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e592b297f0c5 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:mon.b smithi171 running (8m) 47s ago 8m 39.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 2dee2d89540c 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:mon.c smithi186 running (8m) 47s ago 8m 40.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef d82c1bb37bb8 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.0 smithi111 running (8m) 22s ago 8m 54.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 6097849997f3 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.1 smithi171 running (7m) 47s ago 7m 56.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 7845ac9a9838 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.2 smithi186 running (7m) 47s ago 7m 55.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 687b851ce8ef 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.3 smithi111 running (6m) 22s ago 6m 46.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e5f98619ce34 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.4 smithi171 running (6m) 47s ago 6m 52.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef bd3bce09c31c 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.5 smithi186 running (5m) 47s ago 5m 49.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b269345a00d4 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.6 smithi111 running (5m) 22s ago 5m 49.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e9304d1e0f46 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.7 smithi171 running (4m) 47s ago 4m 57.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 92622334e2ff 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.8 smithi186 running (4m) 47s ago 4m 56.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 945875b321e4 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.9 smithi111 running (3m) 22s ago 3m 43.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 76f13029e15f 2024-04-24T22:41:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:osd.10 smithi171 running (3m) 47s ago 3m 45.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef fb3a5439d122 2024-04-24T22:41:20.921 INFO:teuthology.orchestra.run.smithi111.stdout:osd.11 smithi186 running (2m) 47s ago 2m 54.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef cf8a8626f928 2024-04-24T22:41:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 msgr2=0x7f03f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 0x7f03f80a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f04141032f0 tx=0x7f040400b410 comp rx=0 tx=0).stop 2024-04-24T22:41:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 msgr2=0x7f041419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f040000e9d0 tx=0x7f040000ece0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 shutdown_connections 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0414103e30 0x7f04141a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f03f80a2c20 0x7f03f80a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0414102290 0x7f041419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 --2- 172.21.15.111:0/2900039099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0414103490 0x7f041419abe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 >> 172.21.15.111:0/2900039099 conn(0x7f04140fd830 msgr2=0x7f0414074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 shutdown_connections 2024-04-24T22:41:20.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:20.919+0000 7f041a800700 1 -- 172.21.15.111:0/2900039099 wait complete. 2024-04-24T22:41:21.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:20 smithi171 ceph-mon[32337]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:21.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:20 smithi186 ceph-mon[32753]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:21.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T22:41:22.128 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:22 smithi111 ceph-mon[27746]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:23.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:22 smithi111 ceph-mon[27746]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:22 smithi171 ceph-mon[32337]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:22 smithi171 ceph-mon[32337]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:22 smithi186 ceph-mon[32753]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:22 smithi186 ceph-mon[32753]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:24.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.409+0000 7f666a800700 1 -- 172.21.15.111:0/809157181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641056d0 msgr2=0x7f6664105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:24.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.409+0000 7f666a800700 1 --2- 172.21.15.111:0/809157181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641056d0 0x7f6664105b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f664c0099c0 tx=0x7f664c009cd0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 -- 172.21.15.111:0/809157181 shutdown_connections 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 --2- 172.21.15.111:0/809157181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 0x7f666410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 --2- 172.21.15.111:0/809157181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641044d0 0x7f66641048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 --2- 172.21.15.111:0/809157181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641056d0 0x7f6664105b30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 -- 172.21.15.111:0/809157181 >> 172.21.15.111:0/809157181 conn(0x7f66640ffa90 msgr2=0x7f6664101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 -- 172.21.15.111:0/809157181 shutdown_connections 2024-04-24T22:41:24.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.410+0000 7f666a800700 1 -- 172.21.15.111:0/809157181 wait complete. 2024-04-24T22:41:24.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 Processor -- start 2024-04-24T22:41:24.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 -- start start 2024-04-24T22:41:24.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:24.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 0x7f666419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:24.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 0x7f66641a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66641101b0 con 0x7f66641044d0 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6664110030 con 0x7f66641056d0 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.411+0000 7f666a800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6664110330 con 0x7f6664106070 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41124/0 (socket says 172.21.15.111:41124) 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 -- 172.21.15.111:0/2505453211 learned_addr learned my addr 172.21.15.111:0/2505453211 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663e00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 0x7f66641a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f665be00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 0x7f666419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 -- 172.21.15.111:0/2505453211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 msgr2=0x7f66641a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 0x7f66641a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 -- 172.21.15.111:0/2505453211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 msgr2=0x7f666419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 0x7f666419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 -- 172.21.15.111:0/2505453211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66641a23a0 con 0x7f66641044d0 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f665be00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 0x7f666419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.412+0000 7f6663400700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f665400b530 tx=0x7f665400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.413+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654015020 con 0x7f66641044d0 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.413+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6654011d10 con 0x7f66641044d0 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.413+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66641a2690 con 0x7f66641044d0 2024-04-24T22:41:24.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.413+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665400f2e0 con 0x7f66641044d0 2024-04-24T22:41:24.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.413+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66641a2c50 con 0x7f66641044d0 2024-04-24T22:41:24.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.414+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6664068f20 con 0x7f66641044d0 2024-04-24T22:41:24.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.416+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6654018020 con 0x7f66641044d0 2024-04-24T22:41:24.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.418+0000 7f6660c00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 0x7f66480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:24.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.419+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f66540cec50 con 0x7f66641044d0 2024-04-24T22:41:24.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.419+0000 7f665be00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 0x7f66480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.419+0000 7f665be00700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 0x7f66480a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:24.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.422+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f665401f020 con 0x7f66641044d0 2024-04-24T22:41:24.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.656+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6664062670 con 0x7f66480a2c20 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.664+0000 7f6660c00700 1 -- 172.21.15.111:0/2505453211 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6664062670 con 0x7f66480a2c20 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stdout:mds.all 12/12 51s ago 2m smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stdout:mgr 3/3 51s ago 8m smithi111=x;smithi171=y;smithi186=z;count:3 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stdout:mon 3/3 51s ago 9m smithi111:172.21.15.111=a;smithi171:172.21.15.171=b;smithi186:172.21.15.186=c;count:3 2024-04-24T22:41:24.666 INFO:teuthology.orchestra.run.smithi111.stdout:osd 12 51s ago - 2024-04-24T22:41:24.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 msgr2=0x7f66480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:24.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 0x7f66480a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c005fd0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 msgr2=0x7f666419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:24.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f665400b530 tx=0x7f665400b840 comp rx=0 tx=0).stop 2024-04-24T22:41:24.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 shutdown_connections 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6664106070 0x7f66641a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66641044d0 0x7f666419a740 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f66480a2c20 0x7f66480a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 --2- 172.21.15.111:0/2505453211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66641056d0 0x7f666419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 >> 172.21.15.111:0/2505453211 conn(0x7f66640ffa90 msgr2=0x7f6664108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 shutdown_connections 2024-04-24T22:41:24.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:24.667+0000 7f666a800700 1 -- 172.21.15.111:0/2505453211 wait complete. 2024-04-24T22:41:25.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:24 smithi111 ceph-mon[27746]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:25.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:24 smithi171 ceph-mon[32337]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:24 smithi186 ceph-mon[32753]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:25.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T22:41:26.001 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:26.150 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:25 smithi111 ceph-mon[27746]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:26.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:25 smithi171 ceph-mon[32337]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:25 smithi186 ceph-mon[32753]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:27.070 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:26 smithi111 ceph-mon[27746]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:27.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:26 smithi171 ceph-mon[32337]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:27.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:26 smithi186 ceph-mon[32753]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.365+0000 7f0b42000700 1 -- 172.21.15.111:0/1007098879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c103d60 msgr2=0x7f0b3c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.365+0000 7f0b42000700 1 --2- 172.21.15.111:0/1007098879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c103d60 0x7f0b3c10a610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b0d0 tx=0x7f0b3400b3e0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 -- 172.21.15.111:0/1007098879 shutdown_connections 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 --2- 172.21.15.111:0/1007098879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c103d60 0x7f0b3c10a610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 --2- 172.21.15.111:0/1007098879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 --2- 172.21.15.111:0/1007098879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 0x7f0b3c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 -- 172.21.15.111:0/1007098879 >> 172.21.15.111:0/1007098879 conn(0x7f0b3c0fd760 msgr2=0x7f0b3c0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 -- 172.21.15.111:0/1007098879 shutdown_connections 2024-04-24T22:41:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.366+0000 7f0b42000700 1 -- 172.21.15.111:0/1007098879 wait complete. 2024-04-24T22:41:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 Processor -- start 2024-04-24T22:41:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 -- start start 2024-04-24T22:41:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 0x7f0b3c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c103d60 0x7f0b3c1a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c10de10 con 0x7f0b3c1021c0 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c10dc90 con 0x7f0b3c1033c0 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.367+0000 7f0b42000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c10df90 con 0x7f0b3c103d60 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34628/0 (socket says 172.21.15.111:34628) 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 -- 172.21.15.111:0/2377223944 learned_addr learned my addr 172.21.15.111:0/2377223944 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b41600700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c103d60 0x7f0b3c1a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b3be00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 0x7f0b3c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 -- 172.21.15.111:0/2377223944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c103d60 msgr2=0x7f0b3c1a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c103d60 0x7f0b3c1a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 -- 172.21.15.111:0/2377223944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 msgr2=0x7f0b3c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 0x7f0b3c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 -- 172.21.15.111:0/2377223944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c1a2200 con 0x7f0b3c1021c0 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.368+0000 7f0b40c00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e980 tx=0x7f0b3000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.369+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30004230 con 0x7f0b3c1021c0 2024-04-24T22:41:28.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.369+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b300043d0 con 0x7f0b3c1021c0 2024-04-24T22:41:28.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.369+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30003dc0 con 0x7f0b3c1021c0 2024-04-24T22:41:28.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.369+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c1a24f0 con 0x7f0b3c1021c0 2024-04-24T22:41:28.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.369+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c1a2a30 con 0x7f0b3c1021c0 2024-04-24T22:41:28.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.370+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3c068f20 con 0x7f0b3c1021c0 2024-04-24T22:41:28.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.372+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0b3000c810 con 0x7f0b3c1021c0 2024-04-24T22:41:28.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.373+0000 7f0b39600700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 0x7f0b240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.374+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0b300cf2a0 con 0x7f0b3c1021c0 2024-04-24T22:41:28.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.374+0000 7f0b3be00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 0x7f0b240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.374+0000 7f0b3be00700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 0x7f0b240a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009c60 tx=0x7f0b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.378+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b3009c1b0 con 0x7f0b3c1021c0 2024-04-24T22:41:28.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.612+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0b3c075b80 con 0x7f0b240a2ca0 2024-04-24T22:41:28.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.615+0000 7f0b39600700 1 -- 172.21.15.111:0/2377223944 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0b3c075b80 con 0x7f0b240a2ca0 2024-04-24T22:41:28.617 INFO:teuthology.orchestra.run.smithi111.stdout:HOST ADDR LABELS STATUS 2024-04-24T22:41:28.618 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111 172.21.15.111 2024-04-24T22:41:28.618 INFO:teuthology.orchestra.run.smithi111.stdout:smithi171 172.21.15.171 2024-04-24T22:41:28.618 INFO:teuthology.orchestra.run.smithi111.stdout:smithi186 172.21.15.186 2024-04-24T22:41:28.618 INFO:teuthology.orchestra.run.smithi111.stdout:3 hosts in cluster 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 msgr2=0x7f0b240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 0x7f0b240a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009c60 tx=0x7f0b28009260 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 msgr2=0x7f0b3c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e980 tx=0x7f0b3000ec90 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 shutdown_connections 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b3c103d60 0x7f0b3c1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b3c1021c0 0x7f0b3c19a5a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f0b240a2ca0 0x7f0b240a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 --2- 172.21.15.111:0/2377223944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b3c1033c0 0x7f0b3c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.618+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 >> 172.21.15.111:0/2377223944 conn(0x7f0b3c0fd760 msgr2=0x7f0b3c074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.619+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 shutdown_connections 2024-04-24T22:41:28.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:28.619+0000 7f0b42000700 1 -- 172.21.15.111:0/2377223944 wait complete. 2024-04-24T22:41:29.019 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:28 smithi111 ceph-mon[27746]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:29.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:28 smithi171 ceph-mon[32337]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:29.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:28 smithi186 ceph-mon[32753]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:29.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T22:41:29.946 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:30.058 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:29 smithi171 ceph-mon[32337]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:30.092 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:29 smithi111 ceph-mon[27746]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:30.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:29 smithi186 ceph-mon[32753]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:31.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:30 smithi171 ceph-mon[32337]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:31.184 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:30 smithi111 ceph-mon[27746]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:31.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:30 smithi186 ceph-mon[32753]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:32.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 -- 172.21.15.111:0/3975283166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001056d0 msgr2=0x7f9100105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 --2- 172.21.15.111:0/3975283166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001056d0 0x7f9100105b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009920 tx=0x7f90ec009c30 comp rx=0 tx=0).stop 2024-04-24T22:41:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 -- 172.21.15.111:0/3975283166 shutdown_connections 2024-04-24T22:41:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 --2- 172.21.15.111:0/3975283166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f910010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 --2- 172.21.15.111:0/3975283166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91001044d0 0x7f91001048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 --2- 172.21.15.111:0/3975283166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001056d0 0x7f9100105b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 -- 172.21.15.111:0/3975283166 >> 172.21.15.111:0/3975283166 conn(0x7f91000ffa90 msgr2=0x7f9100101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 -- 172.21.15.111:0/3975283166 shutdown_connections 2024-04-24T22:41:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.236+0000 7f9105800700 1 -- 172.21.15.111:0/3975283166 wait complete. 2024-04-24T22:41:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.237+0000 7f9105800700 1 Processor -- start 2024-04-24T22:41:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 -- start start 2024-04-24T22:41:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91001056d0 0x7f910019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f91001100e0 con 0x7f9100106070 2024-04-24T22:41:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f910010ff60 con 0x7f91001044d0 2024-04-24T22:41:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.238+0000 7f9105800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9100110260 con 0x7f91001056d0 2024-04-24T22:41:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90fea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90fea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57636/0 (socket says 172.21.15.111:57636) 2024-04-24T22:41:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90fea00700 1 -- 172.21.15.111:0/1184446858 learned_addr learned my addr 172.21.15.111:0/1184446858 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:32.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34648/0 (socket says 172.21.15.111:34648) 2024-04-24T22:41:32.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 -- 172.21.15.111:0/1184446858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91001056d0 msgr2=0x7f910019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:32.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91001056d0 0x7f910019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 -- 172.21.15.111:0/1184446858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 msgr2=0x7f910019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:32.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 -- 172.21.15.111:0/1184446858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91001a2380 con 0x7f9100106070 2024-04-24T22:41:32.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90fea00700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:32.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.239+0000 7f90ff400700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e970 tx=0x7f90f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:32.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.240+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004220 con 0x7f9100106070 2024-04-24T22:41:32.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.240+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8004d10 con 0x7f9100106070 2024-04-24T22:41:32.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.240+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91001a2670 con 0x7f9100106070 2024-04-24T22:41:32.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.240+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80056b0 con 0x7f9100106070 2024-04-24T22:41:32.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.240+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91001a2bb0 con 0x7f9100106070 2024-04-24T22:41:32.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.241+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9100068f20 con 0x7f9100106070 2024-04-24T22:41:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.243+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f90f8004750 con 0x7f9100106070 2024-04-24T22:41:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.244+0000 7f90f7400700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 0x7f90e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.244+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f90f80ced60 con 0x7f9100106070 2024-04-24T22:41:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.245+0000 7f90fea00700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 0x7f90e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.245+0000 7f90fea00700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 0x7f90e40a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003ef0 tx=0x7f90f000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.249+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90f809bc70 con 0x7f9100106070 2024-04-24T22:41:32.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.480+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 --> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9100062670 con 0x7f90e40a2ca0 2024-04-24T22:41:32.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.484+0000 7f90f7400700 1 -- 172.21.15.111:0/1184446858 <== mgr.14152 v2:172.21.15.111:6800/1872264645 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9100062670 con 0x7f90e40a2ca0 2024-04-24T22:41:32.487 INFO:teuthology.orchestra.run.smithi111.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T22:41:32.487 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T22:41:32.487 INFO:teuthology.orchestra.run.smithi111.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T22:41:32.487 INFO:teuthology.orchestra.run.smithi111.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T22:41:32.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.487+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 msgr2=0x7f90e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.487+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 0x7f90e40a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003ef0 tx=0x7f90f000b040 comp rx=0 tx=0).stop 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.487+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 msgr2=0x7f91001a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.487+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e970 tx=0x7f90f800ec80 comp rx=0 tx=0).stop 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 shutdown_connections 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9100106070 0x7f91001a1ce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f90e40a2ca0 0x7f90e40a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91001044d0 0x7f910019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 --2- 172.21.15.111:0/1184446858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91001056d0 0x7f910019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:32.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 >> 172.21.15.111:0/1184446858 conn(0x7f91000ffa90 msgr2=0x7f91001089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:32.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 shutdown_connections 2024-04-24T22:41:32.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:32.488+0000 7f9105800700 1 -- 172.21.15.111:0/1184446858 wait complete. 2024-04-24T22:41:33.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:32 smithi171 ceph-mon[32337]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:33.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:32 smithi186 ceph-mon[32753]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:33.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:32 smithi111 ceph-mon[27746]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:33.388 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:41:33.399 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:33.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T22:41:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:34.151 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:33 smithi186 ceph-mon[32753]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:34.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:33 smithi171 ceph-mon[32337]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:34.161 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:33 smithi111 ceph-mon[27746]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:35.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:34 smithi171 ceph-mon[32337]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:35.236 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:34 smithi111 ceph-mon[27746]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:35.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:34 smithi186 ceph-mon[32753]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:36.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 -- 172.21.15.111:0/3159526239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94076260 msgr2=0x7fef940746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 --2- 172.21.15.111:0/3159526239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94076260 0x7fef940746c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef84009990 tx=0x7fef84009ca0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 -- 172.21.15.111:0/3159526239 shutdown_connections 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 --2- 172.21.15.111:0/3159526239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9410a170 0x7fef9410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 --2- 172.21.15.111:0/3159526239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94076260 0x7fef940746c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 --2- 172.21.15.111:0/3159526239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef94074c90 0x7fef94075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 -- 172.21.15.111:0/3159526239 >> 172.21.15.111:0/3159526239 conn(0x7fef940fd800 msgr2=0x7fef940ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 -- 172.21.15.111:0/3159526239 shutdown_connections 2024-04-24T22:41:36.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.083+0000 7fef9ac00700 1 -- 172.21.15.111:0/3159526239 wait complete. 2024-04-24T22:41:36.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 Processor -- start 2024-04-24T22:41:36.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 -- start start 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 0x7fef9419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef9410a170 0x7fef941a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef941100f0 con 0x7fef9410a170 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.085+0000 7fef9ac00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef9410ff70 con 0x7fef94074c90 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef9ac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef94110270 con 0x7fef94076260 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef93e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 0x7fef9419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef93e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33910/0 (socket says 172.21.15.111:33910) 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef93e00700 1 -- 172.21.15.111:0/3776165268 learned_addr learned my addr 172.21.15.111:0/3776165268 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:36.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef98c00700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef9410a170 0x7fef941a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:36.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 -- 172.21.15.111:0/3776165268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 msgr2=0x7fef9419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:36.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 -- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef9410a170 msgr2=0x7fef941a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:36.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef9410a170 0x7fef941a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef8b400700 1 -- 172.21.15.111:0/3776165268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef941a2340 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.086+0000 7fef93e00700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef8b400700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 0x7fef9419ac70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00cb70 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c00cd10 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c018700 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef9ac00700 1 -- 172.21.15.111:0/3776165268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef941a2630 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.087+0000 7fef9ac00700 1 -- 172.21.15.111:0/3776165268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a2b70 con 0x7fef94076260 2024-04-24T22:41:36.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.088+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c005330 con 0x7fef94076260 2024-04-24T22:41:36.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.090+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef7c01f020 con 0x7fef94076260 2024-04-24T22:41:36.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.092+0000 7fef91600700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 0x7fef780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:36.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.092+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fef7c0cef30 con 0x7fef94076260 2024-04-24T22:41:36.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.092+0000 7fef93e00700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 0x7fef780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:36.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.093+0000 7fef93e00700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 0x7fef780a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef8400b2e0 tx=0x7fef84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:36.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.096+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fef7c09be40 con 0x7fef94076260 2024-04-24T22:41:36.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.368+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fef6c005e30 con 0x7fef94076260 2024-04-24T22:41:36.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.369+0000 7fef91600700 1 -- 172.21.15.111:0/3776165268 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3054 (secure 0 0 0) 0x7fef7c09b740 con 0x7fef94076260 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout:e23 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout:legacy client fscid: 1 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:36.372 INFO:teuthology.orchestra.run.smithi111.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:fs_name cephfs 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 23 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-04-24T22:39:30.113682+0000 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:modified 2024-04-24T22:39:35.992637+0000 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:tableserver 0 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:root 0 2024-04-24T22:41:36.373 INFO:teuthology.orchestra.run.smithi111.stdout:session_timeout 300 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:session_autoclose 300 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:max_file_size 1099511627776 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:max_xattr_size 65536 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:required_client_features {} 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:last_failure 0 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:last_failure_osd_epoch 0 2024-04-24T22:41:36.374 INFO:teuthology.orchestra.run.smithi111.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:max_mds 3 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:in 0,1,2 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:up {0=24587,1=24574,2=24556} 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:failed 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:damaged 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:stopped 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:data_pools [3] 2024-04-24T22:41:36.375 INFO:teuthology.orchestra.run.smithi111.stdout:metadata_pool 2 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:inline_data disabled 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:balancer 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:bal_rank_mask -1 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:standby_count_wanted 0 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:qdb_cluster leader: 24587 members: 24556,24574,24587 2024-04-24T22:41:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.h{0:24539} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/3780798137,v1:172.21.15.171:6837/3780798137] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.g{1:24574} state up:active seq 5 addr [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c{1:24535} state up:standby-replay seq 1 addr [v2:172.21.15.186:6836/1876976469,v1:172.21.15.186:6837/1876976469] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.i{2:24556} state up:active seq 7 addr [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.f{2:24523} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/964018737,v1:172.21.15.186:6835/964018737] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:36.377 INFO:teuthology.orchestra.run.smithi111.stdout:Standby daemons: 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.j{-1:14394} state up:standby seq 1 addr [v2:172.21.15.111:6834/425814781,v1:172.21.15.111:6835/425814781] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.111:6836/2737045039,v1:172.21.15.111:6837/2737045039] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a{-1:14469} state up:standby seq 1 addr [v2:172.21.15.111:6840/667255252,v1:172.21.15.111:6841/667255252] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.186:6832/1686750740,v1:172.21.15.186:6833/1686750740] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.378 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.k{-1:24464} state up:standby seq 1 addr [v2:172.21.15.171:6832/254707808,v1:172.21.15.171:6833/254707808] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.379 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.e{-1:24506} state up:standby seq 1 addr [v2:172.21.15.171:6834/4258808378,v1:172.21.15.171:6835/4258808378] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:41:36.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 msgr2=0x7fef780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:36.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 0x7fef780a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef8400b2e0 tx=0x7fef84005fd0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 msgr2=0x7fef9419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 0x7fef9419ac70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00ed00 comp rx=0 tx=0).stop 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 shutdown_connections 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef9410a170 0x7fef941a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fef780a2c20 0x7fef780a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef94074c90 0x7fef9419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 --2- 172.21.15.111:0/3776165268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef94076260 0x7fef9419ac70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:36.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 >> 172.21.15.111:0/3776165268 conn(0x7fef940fd800 msgr2=0x7fef94106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:36.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 shutdown_connections 2024-04-24T22:41:36.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:36.372+0000 7fef8a000700 1 -- 172.21.15.111:0/3776165268 wait complete. 2024-04-24T22:41:36.382 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:37.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:36 smithi111 ceph-mon[27746]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:37.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:36 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3776165268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:41:37.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T22:41:37.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:36 smithi171 ceph-mon[32337]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:37.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:36 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3776165268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:41:37.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:36 smithi186 ceph-mon[32753]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:37.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:36 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3776165268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:41:37.610 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:39.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:38 smithi111 ceph-mon[27746]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:39.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:38 smithi171 ceph-mon[32337]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:39.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:38 smithi186 ceph-mon[32753]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 -- 172.21.15.111:0/3388969972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 msgr2=0x7fdd8c10a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 --2- 172.21.15.111:0/3388969972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c10a6f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009fc0 tx=0x7fdd8400b330 comp rx=0 tx=0).stop 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 -- 172.21.15.111:0/3388969972 shutdown_connections 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 --2- 172.21.15.111:0/3388969972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c10a6f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 --2- 172.21.15.111:0/3388969972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 --2- 172.21.15.111:0/3388969972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1034a0 0x7fdd8c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 -- 172.21.15.111:0/3388969972 >> 172.21.15.111:0/3388969972 conn(0x7fdd8c0fd800 msgr2=0x7fdd8c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 -- 172.21.15.111:0/3388969972 shutdown_connections 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.914+0000 7fdd93a00700 1 -- 172.21.15.111:0/3388969972 wait complete. 2024-04-24T22:41:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.915+0000 7fdd93a00700 1 Processor -- start 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 -- start start 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1034a0 0x7fdd8c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10deb0 con 0x7fdd8c1022a0 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10dd30 con 0x7fdd8c103e40 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd93a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10e030 con 0x7fdd8c1034a0 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd91000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd91000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58822/0 (socket says 172.21.15.111:58822) 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd91000700 1 -- 172.21.15.111:0/720559723 learned_addr learned my addr 172.21.15.111:0/720559723 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd91a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.916+0000 7fdd83e00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1034a0 0x7fdd8c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 -- 172.21.15.111:0/720559723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1034a0 msgr2=0x7fdd8c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1034a0 0x7fdd8c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 -- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 msgr2=0x7fdd8c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 -- 172.21.15.111:0/720559723 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c1a22d0 con 0x7fdd8c103e40 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91000700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd91a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c1a1c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd84007c50 tx=0x7fdd84007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401f070 con 0x7fdd8c103e40 2024-04-24T22:41:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd84003e10 con 0x7fdd8c103e40 2024-04-24T22:41:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401b6a0 con 0x7fdd8c103e40 2024-04-24T22:41:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c1a2560 con 0x7fdd8c103e40 2024-04-24T22:41:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.917+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1a29b0 con 0x7fdd8c103e40 2024-04-24T22:41:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.919+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c068f20 con 0x7fdd8c103e40 2024-04-24T22:41:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.920+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdd84022070 con 0x7fdd8c103e40 2024-04-24T22:41:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.922+0000 7fdd8a000700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 0x7fdd680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.922+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdd8400f020 con 0x7fdd8c103e40 2024-04-24T22:41:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.922+0000 7fdd91000700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 0x7fdd680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.923+0000 7fdd91000700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 0x7fdd680a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005950 tx=0x7fdd70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:39.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:39.927+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd84029020 con 0x7fdd8c103e40 2024-04-24T22:41:40.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.154+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fdd8c1a2c60 con 0x7fdd8c103e40 2024-04-24T22:41:40.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.155+0000 7fdd8a000700 1 -- 172.21.15.111:0/720559723 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5303 (secure 0 0 0) 0x7fdd8409d000 con 0x7fdd8c103e40 2024-04-24T22:41:40.157 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 87 2024-04-24T22:41:40.157 INFO:teuthology.orchestra.run.smithi111.stdout:fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:41:40.157 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-04-24T22:30:20.053310+0000 2024-04-24T22:41:40.157 INFO:teuthology.orchestra.run.smithi111.stdout:modified 2024-04-24T22:40:28.695337+0000 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:crush_version 26 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:full_ratio 0.95 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:backfillfull_ratio 0.9 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:nearfull_ratio 0.85 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:require_min_compat_client luminous 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:min_compat_client luminous 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:require_osd_release squid 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:stretch_mode_enabled false 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:max_osd 12 2024-04-24T22:41:40.158 INFO:teuthology.orchestra.run.smithi111.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6802/1011608049,v1:172.21.15.111:6803/1011608049] [v2:172.21.15.111:6804/1011608049,v1:172.21.15.111:6805/1011608049] exists,up 6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1334857356,v1:172.21.15.171:6801/1334857356] [v2:172.21.15.171:6802/1334857356,v1:172.21.15.171:6803/1334857356] exists,up 9336b8fd-028c-49e6-ac02-839e7eb43edf 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/2871610305,v1:172.21.15.186:6801/2871610305] [v2:172.21.15.186:6802/2871610305,v1:172.21.15.186:6803/2871610305] exists,up 43993a4c-fb63-47d7-af12-b6cc7f7c2d9b 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6810/3102831357,v1:172.21.15.111:6811/3102831357] [v2:172.21.15.111:6812/3102831357,v1:172.21.15.111:6813/3102831357] exists,up 4d7cb3cc-81a9-4f99-802b-d29c399eb6b1 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2305118347,v1:172.21.15.171:6809/2305118347] [v2:172.21.15.171:6810/2305118347,v1:172.21.15.171:6811/2305118347] exists,up 7d8ef87e-1192-4e17-8861-c2214d514b34 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/2042071677,v1:172.21.15.186:6809/2042071677] [v2:172.21.15.186:6810/2042071677,v1:172.21.15.186:6811/2042071677] exists,up 1ac00518-3f56-44e1-84fb-9f5f57fb12a3 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6818/525582463,v1:172.21.15.111:6819/525582463] [v2:172.21.15.111:6820/525582463,v1:172.21.15.111:6821/525582463] exists,up e612d700-d87e-4031-b2f9-6455030a282b 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/2409942140,v1:172.21.15.171:6817/2409942140] [v2:172.21.15.171:6818/2409942140,v1:172.21.15.171:6819/2409942140] exists,up ac3e009c-2c29-480c-b226-9df2d6f0d5ed 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/796165885,v1:172.21.15.186:6817/796165885] [v2:172.21.15.186:6818/796165885,v1:172.21.15.186:6819/796165885] exists,up 789c1edf-d22f-4a42-9b25-02cedbccadd4 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.9 up in weight 1 up_from 67 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6826/637881617,v1:172.21.15.111:6827/637881617] [v2:172.21.15.111:6828/637881617,v1:172.21.15.111:6829/637881617] exists,up aa3d5a06-570a-4e45-874e-c64a64f66b9b 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/4154918762,v1:172.21.15.171:6825/4154918762] [v2:172.21.15.171:6826/4154918762,v1:172.21.15.171:6827/4154918762] exists,up 13df39cd-031b-431a-90a9-76d1c09f1922 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/33009577,v1:172.21.15.186:6825/33009577] [v2:172.21.15.186:6826/33009577,v1:172.21.15.186:6827/33009577] exists,up add0e95d-86a9-4c00-8c5e-e775d9afabbf 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.b [8,11] 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.10 [2,4] 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.13 [2,11] 2024-04-24T22:41:40.159 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.e [5,9] 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.12 [3,9] 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.23 [4,7] 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.31 [4,8] 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.36 [4,9] 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/2026940770 expires 2024-04-25T22:31:27.425198+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/1436625216 expires 2024-04-25T22:31:27.425198+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6801/3447332391 expires 2024-04-25T22:31:27.425198+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6800/2142544018 expires 2024-04-25T22:30:53.108811+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/1327559640 expires 2024-04-25T22:30:53.108811+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/4190678364 expires 2024-04-25T22:30:53.108811+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/449834621 expires 2024-04-25T22:31:27.425198+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6801/2142544018 expires 2024-04-25T22:30:53.108811+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/4240227320 expires 2024-04-25T22:30:53.108811+0000 2024-04-24T22:41:40.160 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6800/3447332391 expires 2024-04-25T22:31:27.425198+0000 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.159+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 msgr2=0x7fdd680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.159+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 0x7fdd680a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005950 tx=0x7fdd70009260 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.159+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 msgr2=0x7fdd8c1a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c1a1c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd84007c50 tx=0x7fdd84007c80 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 shutdown_connections 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fdd680a2c20 0x7fdd680a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd8c103e40 0x7fdd8c1a1c30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd8c1022a0 0x7fdd8c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 --2- 172.21.15.111:0/720559723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd8c1034a0 0x7fdd8c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 >> 172.21.15.111:0/720559723 conn(0x7fdd8c0fd800 msgr2=0x7fdd8c074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:40.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 shutdown_connections 2024-04-24T22:41:40.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:40.160+0000 7fdd93a00700 1 -- 172.21.15.111:0/720559723 wait complete. 2024-04-24T22:41:40.986 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T22:41:40.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:41.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:40 smithi171 ceph-mon[32337]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:41.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:40 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/720559723' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T22:41:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 -- 172.21.15.111:0/1876357535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee000ffa50 msgr2=0x7fee000ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 --2- 172.21.15.111:0/1876357535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee000ffa50 0x7fee000ffe70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009900 tx=0x7fedf002a410 comp rx=0 tx=0).stop 2024-04-24T22:41:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 -- 172.21.15.111:0/1876357535 shutdown_connections 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 --2- 172.21.15.111:0/1876357535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee00101170 0x7fee001019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 --2- 172.21.15.111:0/1876357535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001007b0 0x7fee00100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 --2- 172.21.15.111:0/1876357535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee000ffa50 0x7fee000ffe70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 -- 172.21.15.111:0/1876357535 >> 172.21.15.111:0/1876357535 conn(0x7fee000fb380 msgr2=0x7fee000fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 -- 172.21.15.111:0/1876357535 shutdown_connections 2024-04-24T22:41:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.206+0000 7fee04c00640 1 -- 172.21.15.111:0/1876357535 wait complete. 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.207+0000 7fee04c00640 1 Processor -- start 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 -- start start 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee000ffa50 0x7fee00199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 0x7fee001a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010f650 con 0x7fee001007b0 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010f4d0 con 0x7fee00101170 2024-04-24T22:41:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fee04c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010f7d0 con 0x7fee000ffa50 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fedfd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fedfd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58832/0 (socket says 172.21.15.111:58832) 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fedfd400640 1 -- 172.21.15.111:0/495424731 learned_addr learned my addr 172.21.15.111:0/495424731 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fedfe800640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 0x7fee001a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.208+0000 7fedfde00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee000ffa50 0x7fee00199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 -- 172.21.15.111:0/495424731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee000ffa50 msgr2=0x7fee00199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee000ffa50 0x7fee00199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 -- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 msgr2=0x7fee0019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 -- 172.21.15.111:0/495424731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee001a18d0 con 0x7fee00101170 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfd400640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedfe800640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 0x7fee001a1230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ec50 tx=0x7fedf800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8003b70 con 0x7fee00101170 2024-04-24T22:41:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee001a1bc0 con 0x7fee00101170 2024-04-24T22:41:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8003d10 con 0x7fee00101170 2024-04-24T22:41:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8010890 con 0x7fee00101170 2024-04-24T22:41:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.209+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee001a2090 con 0x7fee00101170 2024-04-24T22:41:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.210+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0010b560 con 0x7fee00101170 2024-04-24T22:41:41.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.212+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fedf8004b40 con 0x7fee00101170 2024-04-24T22:41:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.214+0000 7fedf6a00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 0x7fede40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.214+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fedf80cf5c0 con 0x7fee00101170 2024-04-24T22:41:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.214+0000 7fedfde00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 0x7fede40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.215+0000 7fedfde00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 0x7fede40a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031440 tx=0x7fedf00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.216+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedf809c4d0 con 0x7fee00101170 2024-04-24T22:41:41.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:40 smithi186 ceph-mon[32753]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:41.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:40 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/720559723' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T22:41:41.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:40 smithi111 ceph-mon[27746]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T22:41:41.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:40 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/720559723' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T22:41:41.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.418+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee0002d3c0 con 0x7fee00101170 2024-04-24T22:41:41.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.420+0000 7fedf6a00640 1 -- 172.21.15.111:0/495424731 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7fedf809bdd0 con 0x7fee00101170 2024-04-24T22:41:41.422 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:41.423 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 msgr2=0x7fede40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 0x7fede40a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031440 tx=0x7fedf00315b0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 msgr2=0x7fee001a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 0x7fee001a1230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ec50 tx=0x7fedf800c6a0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 shutdown_connections 2024-04-24T22:41:41.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fede40a2b90 0x7fede40a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee00101170 0x7fee001a1230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee001007b0 0x7fee0019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 --2- 172.21.15.111:0/495424731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee000ffa50 0x7fee00199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 >> 172.21.15.111:0/495424731 conn(0x7fee000fb380 msgr2=0x7fee000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.423+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 shutdown_connections 2024-04-24T22:41:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.424+0000 7fee04c00640 1 -- 172.21.15.111:0/495424731 wait complete. 2024-04-24T22:41:41.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:41:41.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 -- 172.21.15.111:0/2889859814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 msgr2=0x7f04c4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 --2- 172.21.15.111:0/2889859814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4101cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009900 tx=0x7f04ac02a410 comp rx=0 tx=0).stop 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 -- 172.21.15.111:0/2889859814 shutdown_connections 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 --2- 172.21.15.111:0/2889859814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4102230 0x7f04c4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 --2- 172.21.15.111:0/2889859814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4101cf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 --2- 172.21.15.111:0/2889859814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4100680 0x7f04c4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 -- 172.21.15.111:0/2889859814 >> 172.21.15.111:0/2889859814 conn(0x7f04c40fbbf0 msgr2=0x7f04c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 -- 172.21.15.111:0/2889859814 shutdown_connections 2024-04-24T22:41:41.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.643+0000 7f04ca000640 1 -- 172.21.15.111:0/2889859814 wait complete. 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 Processor -- start 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 -- start start 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4100680 0x7f04c4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 0x7f04c419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04c410c350 con 0x7f04c4101870 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04c410c1d0 con 0x7f04c4100680 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.644+0000 7f04ca000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04c410c4d0 con 0x7f04c4102230 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58850/0 (socket says 172.21.15.111:58850) 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 -- 172.21.15.111:0/1217003763 learned_addr learned my addr 172.21.15.111:0/1217003763 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04c3400640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4100680 0x7f04c4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04c3e00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 0x7f04c419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 -- 172.21.15.111:0/1217003763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 msgr2=0x7f04c419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 0x7f04c419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 -- 172.21.15.111:0/1217003763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4100680 msgr2=0x7f04c4197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4100680 0x7f04c4197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 -- 172.21.15.111:0/1217003763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c419f770 con 0x7f04c4101870 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04c3e00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 0x7f04c419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04bbe00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04ac031440 tx=0x7f04ac032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.645+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac03f030 con 0x7f04c4101870 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.646+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c419fa00 con 0x7f04c4101870 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.646+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac003ce0 con 0x7f04c4101870 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.646+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac049060 con 0x7f04c4101870 2024-04-24T22:41:41.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.646+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c419ff40 con 0x7f04c4101870 2024-04-24T22:41:41.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.647+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c41a0110 con 0x7f04c4101870 2024-04-24T22:41:41.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.649+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f04ac042030 con 0x7f04c4101870 2024-04-24T22:41:41.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.650+0000 7f04c0c00640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 0x7f04a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:41.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.650+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f04ac0ef950 con 0x7f04c4101870 2024-04-24T22:41:41.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.650+0000 7f04c3400640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 0x7f04a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:41.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.651+0000 7f04c3400640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 0x7f04a80a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04c41016d0 tx=0x7f04b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:41.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.655+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f04ac0bc860 con 0x7f04c4101870 2024-04-24T22:41:41.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.856+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04c41a04a0 con 0x7f04c4101870 2024-04-24T22:41:41.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.857+0000 7f04c0c00640 1 -- 172.21.15.111:0/1217003763 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f04ac0bc160 con 0x7f04c4101870 2024-04-24T22:41:41.859 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:41:41.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 msgr2=0x7f04a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 0x7f04a80a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04c41016d0 tx=0x7f04b4005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 msgr2=0x7f04c4198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04ac031440 tx=0x7f04ac032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 shutdown_connections 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f04a80a2b90 0x7f04a80a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c4102230 0x7f04c419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c4101870 0x7f04c4198090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 --2- 172.21.15.111:0/1217003763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04c4100680 0x7f04c4197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.860+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 >> 172.21.15.111:0/1217003763 conn(0x7f04c40fbbf0 msgr2=0x7f04c40689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.861+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 shutdown_connections 2024-04-24T22:41:41.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:41.861+0000 7f04ca000640 1 -- 172.21.15.111:0/1217003763 wait complete. 2024-04-24T22:41:41.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:41:42.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.075+0000 7f265cc00640 1 -- 172.21.15.111:0/4023407933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26581020e0 msgr2=0x7f26581044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.075+0000 7f265cc00640 1 --2- 172.21.15.111:0/4023407933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26581020e0 0x7f26581044d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f264802a410 comp rx=0 tx=0).stop 2024-04-24T22:41:42.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 -- 172.21.15.111:0/4023407933 shutdown_connections 2024-04-24T22:41:42.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 --2- 172.21.15.111:0/4023407933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2658107340 0x7f2658109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 --2- 172.21.15.111:0/4023407933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658104a10 0x7f2658106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 --2- 172.21.15.111:0/4023407933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26581020e0 0x7f26581044d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 -- 172.21.15.111:0/4023407933 >> 172.21.15.111:0/4023407933 conn(0x7f26580fbaf0 msgr2=0x7f26580fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:42.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 -- 172.21.15.111:0/4023407933 shutdown_connections 2024-04-24T22:41:42.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.076+0000 7f265cc00640 1 -- 172.21.15.111:0/4023407933 wait complete. 2024-04-24T22:41:42.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.077+0000 7f265cc00640 1 Processor -- start 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.077+0000 7f265cc00640 1 -- start start 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.077+0000 7f265cc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.077+0000 7f265cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2658104a10 0x7f2658197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f265cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658107340 0x7f265819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f265cc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d340 con 0x7f26581020e0 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f265cc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d1c0 con 0x7f2658104a10 2024-04-24T22:41:42.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f265cc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d4c0 con 0x7f2658107340 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58860/0 (socket says 172.21.15.111:58860) 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 -- 172.21.15.111:0/579179985 learned_addr learned my addr 172.21.15.111:0/579179985 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657e00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658107340 0x7f265819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f264ea00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2658104a10 0x7f2658197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 -- 172.21.15.111:0/579179985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658107340 msgr2=0x7f265819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658107340 0x7f265819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 -- 172.21.15.111:0/579179985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2658104a10 msgr2=0x7f2658197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2658104a10 0x7f2658197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.078+0000 7f2657400640 1 -- 172.21.15.111:0/579179985 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265819f5b0 con 0x7f26581020e0 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f2657400640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2648031440 tx=0x7f2648032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264803f030 con 0x7f26581020e0 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648003ce0 con 0x7f26581020e0 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265819f840 con 0x7f26581020e0 2024-04-24T22:41:42.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035070 con 0x7f26581020e0 2024-04-24T22:41:42.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.079+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265819fd30 con 0x7f26581020e0 2024-04-24T22:41:42.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.080+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26580fd5e0 con 0x7f26581020e0 2024-04-24T22:41:42.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.082+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2648042030 con 0x7f26581020e0 2024-04-24T22:41:42.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.084+0000 7f2654c00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 0x7f263c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.084+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f26480ef5c0 con 0x7f26581020e0 2024-04-24T22:41:42.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.084+0000 7f264ea00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 0x7f263c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.085+0000 7f264ea00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 0x7f263c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2658198ea0 tx=0x7f264000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.088+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26480bbe80 con 0x7f26581020e0 2024-04-24T22:41:42.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:41 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/495424731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:42.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:41 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/495424731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:42.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:41 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/495424731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:42.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.290+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2658062340 con 0x7f26581020e0 2024-04-24T22:41:42.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.291+0000 7f2654c00640 1 -- 172.21.15.111:0/579179985 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f26480c0180 con 0x7f26581020e0 2024-04-24T22:41:42.293 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:41:42.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 msgr2=0x7f263c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 0x7f263c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2658198ea0 tx=0x7f264000b410 comp rx=0 tx=0).stop 2024-04-24T22:41:42.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 msgr2=0x7f2658197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2648031440 tx=0x7f2648032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 shutdown_connections 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f263c0a2b90 0x7f263c0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2658107340 0x7f265819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2658104a10 0x7f2658197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 --2- 172.21.15.111:0/579179985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26581020e0 0x7f2658197990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 >> 172.21.15.111:0/579179985 conn(0x7f26580fbaf0 msgr2=0x7f2658102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 shutdown_connections 2024-04-24T22:41:42.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.294+0000 7f265cc00640 1 -- 172.21.15.111:0/579179985 wait complete. 2024-04-24T22:41:42.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:41:42.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.511+0000 7f65d2c00640 1 -- 172.21.15.111:0/2726914920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc103880 msgr2=0x7f65cc103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.511+0000 7f65d2c00640 1 --2- 172.21.15.111:0/2726914920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc103880 0x7f65cc103ca0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009900 tx=0x7f65c002a410 comp rx=0 tx=0).stop 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 -- 172.21.15.111:0/2726914920 shutdown_connections 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 --2- 172.21.15.111:0/2726914920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc105430 0x7f65cc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 --2- 172.21.15.111:0/2726914920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 0x7f65cc104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 --2- 172.21.15.111:0/2726914920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc103880 0x7f65cc103ca0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 -- 172.21.15.111:0/2726914920 >> 172.21.15.111:0/2726914920 conn(0x7f65cc0fedf0 msgr2=0x7f65cc101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 -- 172.21.15.111:0/2726914920 shutdown_connections 2024-04-24T22:41:42.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.512+0000 7f65d2c00640 1 -- 172.21.15.111:0/2726914920 wait complete. 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 Processor -- start 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 -- start start 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 0x7f65cc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc105430 0x7f65cc1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10f540 con 0x7f65cc105430 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10f3c0 con 0x7f65cc103880 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.513+0000 7f65d2c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10f6c0 con 0x7f65cc104a70 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc105430 0x7f65cc1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33990/0 (socket says 172.21.15.111:33990) 2024-04-24T22:41:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 -- 172.21.15.111:0/839587569 learned_addr learned my addr 172.21.15.111:0/839587569 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d0e00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 0x7f65cc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 -- 172.21.15.111:0/839587569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 msgr2=0x7f65cc19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 0x7f65cc19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 -- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc105430 msgr2=0x7f65cc1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc105430 0x7f65cc1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 -- 172.21.15.111:0/839587569 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc1a1800 con 0x7f65cc103880 2024-04-24T22:41:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65d1800640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65c0031440 tx=0x7f65c0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.514+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c003f030 con 0x7f65cc103880 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.515+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0003ce0 con 0x7f65cc103880 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.515+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc1a1a90 con 0x7f65cc103880 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.515+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0035070 con 0x7f65cc103880 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.515+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a1fd0 con 0x7f65cc103880 2024-04-24T22:41:42.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.516+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc10aba0 con 0x7f65cc103880 2024-04-24T22:41:42.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.518+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f65c0042030 con 0x7f65cc103880 2024-04-24T22:41:42.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.519+0000 7f65ca000640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 0x7f65b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.519+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f65c00ef8c0 con 0x7f65cc103880 2024-04-24T22:41:42.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.519+0000 7f65d0e00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 0x7f65b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.520+0000 7f65d0e00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 0x7f65b40a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f65cc19b0f0 tx=0x7f65bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.521+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f65c00bc7d0 con 0x7f65cc103880 2024-04-24T22:41:42.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.721+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65cc02d3c0 con 0x7f65cc103880 2024-04-24T22:41:42.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.722+0000 7f65ca000640 1 -- 172.21.15.111:0/839587569 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f65c00bc0d0 con 0x7f65cc103880 2024-04-24T22:41:42.724 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:41:42.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 msgr2=0x7f65b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 0x7f65b40a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f65cc19b0f0 tx=0x7f65bc005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 msgr2=0x7f65cc199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65c0031440 tx=0x7f65c0032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 shutdown_connections 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f65b40a2c10 0x7f65b40a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65cc105430 0x7f65cc1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65cc104a70 0x7f65cc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 --2- 172.21.15.111:0/839587569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65cc103880 0x7f65cc199be0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.725+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 >> 172.21.15.111:0/839587569 conn(0x7f65cc0fedf0 msgr2=0x7f65cc107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.726+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 shutdown_connections 2024-04-24T22:41:42.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.726+0000 7f65d2c00640 1 -- 172.21.15.111:0/839587569 wait complete. 2024-04-24T22:41:42.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 -- 172.21.15.111:0/2630486853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c981039c0 msgr2=0x7f4c98103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 --2- 172.21.15.111:0/2630486853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c981039c0 0x7f4c98103de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009900 tx=0x7f4c8802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 -- 172.21.15.111:0/2630486853 shutdown_connections 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 --2- 172.21.15.111:0/2630486853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98105570 0x7f4c9810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 --2- 172.21.15.111:0/2630486853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c98104bb0 0x7f4c98105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 --2- 172.21.15.111:0/2630486853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c981039c0 0x7f4c98103de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 -- 172.21.15.111:0/2630486853 >> 172.21.15.111:0/2630486853 conn(0x7f4c980fef30 msgr2=0x7f4c98101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 -- 172.21.15.111:0/2630486853 shutdown_connections 2024-04-24T22:41:42.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.948+0000 7f4c9f600640 1 -- 172.21.15.111:0/2630486853 wait complete. 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.949+0000 7f4c9f600640 1 Processor -- start 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.949+0000 7f4c9f600640 1 -- start start 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 0x7f4c9819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c98105570 0x7f4c981a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f6e0 con 0x7f4c981039c0 2024-04-24T22:41:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f560 con 0x7f4c98104bb0 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9f600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f860 con 0x7f4c98105570 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58890/0 (socket says 172.21.15.111:58890) 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 -- 172.21.15.111:0/3772665294 learned_addr learned my addr 172.21.15.111:0/3772665294 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c8fe00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 0x7f4c9819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9d600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c98105570 0x7f4c981a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 -- 172.21.15.111:0/3772665294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c98105570 msgr2=0x7f4c981a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c98105570 0x7f4c981a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 -- 172.21.15.111:0/3772665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 msgr2=0x7f4c9819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 0x7f4c9819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.950+0000 7f4c9cc00640 1 -- 172.21.15.111:0/3772665294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c981a1970 con 0x7f4c981039c0 2024-04-24T22:41:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c8fe00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 0x7f4c9819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:42.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c9cc00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c88031440 tx=0x7f4c88032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8803f030 con 0x7f4c981039c0 2024-04-24T22:41:42.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c88003ce0 con 0x7f4c981039c0 2024-04-24T22:41:42.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c981a1c00 con 0x7f4c981039c0 2024-04-24T22:41:42.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88035070 con 0x7f4c981039c0 2024-04-24T22:41:42.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.951+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c981a2140 con 0x7f4c981039c0 2024-04-24T22:41:42.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.952+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9810ad30 con 0x7f4c981039c0 2024-04-24T22:41:42.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.954+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4c88042030 con 0x7f4c981039c0 2024-04-24T22:41:42.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.956+0000 7f4c96000640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 0x7f4c7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.956+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4c880ef4e0 con 0x7f4c981039c0 2024-04-24T22:41:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.956+0000 7f4c8fe00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 0x7f4c7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:42.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.957+0000 7f4c8fe00640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 0x7f4c7c0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c9819b260 tx=0x7f4c80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:42.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:42.960+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c880bbda0 con 0x7f4c981039c0 2024-04-24T22:41:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:42 smithi111 ceph-mon[27746]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:42 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1217003763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:42 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/579179985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:42 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/839587569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:42 smithi171 ceph-mon[32337]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:42 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1217003763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:42 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/579179985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:42 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/839587569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.199+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c980623e0 con 0x7f4c981039c0 2024-04-24T22:41:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.201+0000 7f4c96000640 1 -- 172.21.15.111:0/3772665294 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7f4c880c00a0 con 0x7f4c981039c0 2024-04-24T22:41:43.203 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:43.204 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:41:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 msgr2=0x7f4c7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 0x7f4c7c0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c9819b260 tx=0x7f4c80005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 msgr2=0x7f4c98199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c88031440 tx=0x7f4c88032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 shutdown_connections 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f4c7c0a2c10 0x7f4c7c0a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c98105570 0x7f4c981a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c98104bb0 0x7f4c9819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 --2- 172.21.15.111:0/3772665294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c981039c0 0x7f4c98199d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 >> 172.21.15.111:0/3772665294 conn(0x7f4c980fef30 msgr2=0x7f4c98107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 shutdown_connections 2024-04-24T22:41:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.204+0000 7f4c9f600640 1 -- 172.21.15.111:0/3772665294 wait complete. 2024-04-24T22:41:43.224 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:43.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:42 smithi186 ceph-mon[32753]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T22:41:43.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:42 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1217003763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:42 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/579179985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:42 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/839587569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 -- 172.21.15.111:0/3268516624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 msgr2=0x7f2cf0107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/3268516624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf0107fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009900 tx=0x7f2ce402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 -- 172.21.15.111:0/3268516624 shutdown_connections 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/3268516624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0109770 0x7f2cf0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/3268516624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0108db0 0x7f2cf0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/3268516624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf0107fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 -- 172.21.15.111:0/3268516624 >> 172.21.15.111:0/3268516624 conn(0x7f2cf0076b80 msgr2=0x7f2cf0078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:43.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 -- 172.21.15.111:0/3268516624 shutdown_connections 2024-04-24T22:41:43.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.420+0000 7f2cf6a00640 1 -- 172.21.15.111:0/3268516624 wait complete. 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 Processor -- start 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 -- start start 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0108db0 0x7f2cf019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 0x7f2cf01a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0113810 con 0x7f2cf0109770 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0113690 con 0x7f2cf0107bc0 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.421+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0113990 con 0x7f2cf0108db0 2024-04-24T22:41:43.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 0x7f2cf01a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 0x7f2cf01a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58898/0 (socket says 172.21.15.111:58898) 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34022/0 (socket says 172.21.15.111:34022) 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf4c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0108db0 0x7f2cf019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf6000640 1 -- 172.21.15.111:0/1048197551 learned_addr learned my addr 172.21.15.111:0/1048197551 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 -- 172.21.15.111:0/1048197551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0108db0 msgr2=0x7f2cf019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0108db0 0x7f2cf019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 -- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 msgr2=0x7f2cf01a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 0x7f2cf01a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.422+0000 7f2cf5600640 1 -- 172.21.15.111:0/1048197551 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a5b40 con 0x7f2cf0107bc0 2024-04-24T22:41:43.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cf5600640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009230 tx=0x7f2ce40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:43.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce403f070 con 0x7f2cf0107bc0 2024-04-24T22:41:43.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce403b460 con 0x7f2cf0107bc0 2024-04-24T22:41:43.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4035070 con 0x7f2cf0107bc0 2024-04-24T22:41:43.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a5dd0 con 0x7f2cf0107bc0 2024-04-24T22:41:43.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.423+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a6180 con 0x7f2cf0107bc0 2024-04-24T22:41:43.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.424+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf010efd0 con 0x7f2cf0107bc0 2024-04-24T22:41:43.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.426+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2ce403b910 con 0x7f2cf0107bc0 2024-04-24T22:41:43.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.427+0000 7f2cee000640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 0x7f2cd80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.427+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2ce40ef720 con 0x7f2cf0107bc0 2024-04-24T22:41:43.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.428+0000 7f2cf4c00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 0x7f2cd80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.428+0000 7f2cf4c00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 0x7f2cd80a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cf019f430 tx=0x7f2cdc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:43.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.430+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ce40bbfe0 con 0x7f2cf0107bc0 2024-04-24T22:41:43.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.632+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cf0062440 con 0x7f2cf0107bc0 2024-04-24T22:41:43.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.634+0000 7f2cee000640 1 -- 172.21.15.111:0/1048197551 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7f2ce40c02e0 con 0x7f2cf0107bc0 2024-04-24T22:41:43.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:43.636 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.636+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 msgr2=0x7f2cd80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.636+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 0x7f2cd80a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cf019f430 tx=0x7f2cdc00b390 comp rx=0 tx=0).stop 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.636+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 msgr2=0x7f2cf019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.636+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009230 tx=0x7f2ce40043b0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 shutdown_connections 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f2cd80a2b90 0x7f2cd80a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cf0109770 0x7f2cf01a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf0108db0 0x7f2cf019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 --2- 172.21.15.111:0/1048197551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cf0107bc0 0x7f2cf019df20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 >> 172.21.15.111:0/1048197551 conn(0x7f2cf0076b80 msgr2=0x7f2cf010c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:43.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 shutdown_connections 2024-04-24T22:41:43.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.637+0000 7f2cf6a00640 1 -- 172.21.15.111:0/1048197551 wait complete. 2024-04-24T22:41:43.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:41:43.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:41:43.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 -- 172.21.15.111:0/2721117741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 msgr2=0x7ffad4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 --2- 172.21.15.111:0/2721117741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad4101bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009900 tx=0x7ffabc02a410 comp rx=0 tx=0).stop 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 -- 172.21.15.111:0/2721117741 shutdown_connections 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 --2- 172.21.15.111:0/2721117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad4108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 --2- 172.21.15.111:0/2721117741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad4101bc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 --2- 172.21.15.111:0/2721117741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 -- 172.21.15.111:0/2721117741 >> 172.21.15.111:0/2721117741 conn(0x7ffad40fbae0 msgr2=0x7ffad40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 -- 172.21.15.111:0/2721117741 shutdown_connections 2024-04-24T22:41:43.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.853+0000 7ffad8c00640 1 -- 172.21.15.111:0/2721117741 wait complete. 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.854+0000 7ffad8c00640 1 Processor -- start 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.854+0000 7ffad8c00640 1 -- start start 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad41a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c230 con 0x7ffad4102100 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c0b0 con 0x7ffad4100550 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad8c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c3b0 con 0x7ffad4101740 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad41a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34052/0 (socket says 172.21.15.111:34052) 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3400640 1 -- 172.21.15.111:0/1949303842 learned_addr learned my addr 172.21.15.111:0/1949303842 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffacbe00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 -- 172.21.15.111:0/1949303842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 msgr2=0x7ffad419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 -- 172.21.15.111:0/1949303842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 msgr2=0x7ffad4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3e00640 1 -- 172.21.15.111:0/1949303842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad41a1750 con 0x7ffad4102100 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.855+0000 7ffad3400640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:43.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad3e00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad41a10e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009fd0 tx=0x7ffacc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00ce20 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc005020 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc010720 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a1a40 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.856+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a1f10 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.857+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad41a2140 con 0x7ffad4102100 2024-04-24T22:41:43.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.859+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffacc005540 con 0x7ffad4102100 2024-04-24T22:41:43.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.860+0000 7ffad0c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 0x7ffab80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:43.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.860+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffacc0cf7a0 con 0x7ffad4102100 2024-04-24T22:41:43.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.861+0000 7ffad3400640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 0x7ffab80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:43.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.861+0000 7ffad3400640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 0x7ffab80a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffac40058c0 tx=0x7ffac4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:43.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:43.866+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffacc09c6b0 con 0x7ffad4102100 2024-04-24T22:41:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3772665294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:43 smithi111 ceph-mon[27746]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:43 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1048197551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.051+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffad41a2310 con 0x7ffad4102100 2024-04-24T22:41:44.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.052+0000 7ffad0c00640 1 -- 172.21.15.111:0/1949303842 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7ffacc09bfb0 con 0x7ffad4102100 2024-04-24T22:41:44.054 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 msgr2=0x7ffab80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 0x7ffab80a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffac40058c0 tx=0x7ffac4005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 msgr2=0x7ffad41a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad41a10e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009fd0 tx=0x7ffacc00ef10 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 shutdown_connections 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7ffab80a2b90 0x7ffab80a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4102100 0x7ffad41a10e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad4101740 0x7ffad419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 --2- 172.21.15.111:0/1949303842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100550 0x7ffad4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 >> 172.21.15.111:0/1949303842 conn(0x7ffad40fbae0 msgr2=0x7ffad40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:44.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 shutdown_connections 2024-04-24T22:41:44.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.055+0000 7ffad8c00640 1 -- 172.21.15.111:0/1949303842 wait complete. 2024-04-24T22:41:44.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:41:44.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3772665294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:43 smithi171 ceph-mon[32337]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:44.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:43 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1048197551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3772665294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:43 smithi186 ceph-mon[32753]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T22:41:44.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:43 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1048197551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 -- 172.21.15.111:0/1876663637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c074120 msgr2=0x7fc62c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 --2- 172.21.15.111:0/1876663637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c074120 0x7fc62c074580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc614009860 tx=0x7fc61402a3d0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 -- 172.21.15.111:0/1876663637 shutdown_connections 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 --2- 172.21.15.111:0/1876663637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c109520 0x7fc62c10b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 --2- 172.21.15.111:0/1876663637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c074120 0x7fc62c074580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 --2- 172.21.15.111:0/1876663637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 -- 172.21.15.111:0/1876663637 >> 172.21.15.111:0/1876663637 conn(0x7fc62c0fcbe0 msgr2=0x7fc62c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:44.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 -- 172.21.15.111:0/1876663637 shutdown_connections 2024-04-24T22:41:44.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.271+0000 7fc631200640 1 -- 172.21.15.111:0/1876663637 wait complete. 2024-04-24T22:41:44.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.272+0000 7fc631200640 1 Processor -- start 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.272+0000 7fc631200640 1 -- start start 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 0x7fc62c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10f540 con 0x7fc62c074120 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10f3c0 con 0x7fc62c109520 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc631200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10f6c0 con 0x7fc62c075700 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58932/0 (socket says 172.21.15.111:58932) 2024-04-24T22:41:44.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 -- 172.21.15.111:0/1092238906 learned_addr learned my addr 172.21.15.111:0/1092238906 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62be00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 0x7fc62c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc623e00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 -- 172.21.15.111:0/1092238906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 msgr2=0x7fc62c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 -- 172.21.15.111:0/1092238906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 msgr2=0x7fc62c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 0x7fc62c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc62b400640 1 -- 172.21.15.111:0/1092238906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c1a1790 con 0x7fc62c074120 2024-04-24T22:41:44.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.273+0000 7fc623e00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc62be00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 0x7fc62c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc62b400640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e9f0 tx=0x7fc61c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00cd80 con 0x7fc62c074120 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c1a1a80 con 0x7fc62c074120 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c005500 con 0x7fc62c074120 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c010690 con 0x7fc62c074120 2024-04-24T22:41:44.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.274+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1a1fd0 con 0x7fc62c074120 2024-04-24T22:41:44.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.276+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c1082e0 con 0x7fc62c074120 2024-04-24T22:41:44.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.277+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc61c0046c0 con 0x7fc62c074120 2024-04-24T22:41:44.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.280+0000 7fc628c00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 0x7fc6100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.280+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc61c0cf5f0 con 0x7fc62c074120 2024-04-24T22:41:44.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.280+0000 7fc623e00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 0x7fc6100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.281+0000 7fc623e00640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 0x7fc6100a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6140095f0 tx=0x7fc614005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.284+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc61c09c500 con 0x7fc62c074120 2024-04-24T22:41:44.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.483+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc62c062340 con 0x7fc62c074120 2024-04-24T22:41:44.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.484+0000 7fc628c00640 1 -- 172.21.15.111:0/1092238906 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc61c09be00 con 0x7fc62c074120 2024-04-24T22:41:44.486 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.486+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 msgr2=0x7fc6100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.486+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 0x7fc6100a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6140095f0 tx=0x7fc614005d20 comp rx=0 tx=0).stop 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.486+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 msgr2=0x7fc62c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.486+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e9f0 tx=0x7fc61c00edf0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 shutdown_connections 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc6100a2b90 0x7fc6100a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c109520 0x7fc62c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc62c075700 0x7fc62c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 --2- 172.21.15.111:0/1092238906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc62c074120 0x7fc62c199b70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 >> 172.21.15.111:0/1092238906 conn(0x7fc62c0fcbe0 msgr2=0x7fc62c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 shutdown_connections 2024-04-24T22:41:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.487+0000 7fc631200640 1 -- 172.21.15.111:0/1092238906 wait complete. 2024-04-24T22:41:44.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:41:44.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 -- 172.21.15.111:0/3557877225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c103880 msgr2=0x7f351c103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 --2- 172.21.15.111:0/3557877225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c103880 0x7f351c103ca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f350c009900 tx=0x7f350c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 -- 172.21.15.111:0/3557877225 shutdown_connections 2024-04-24T22:41:44.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 --2- 172.21.15.111:0/3557877225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c105430 0x7f351c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 --2- 172.21.15.111:0/3557877225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 0x7f351c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 --2- 172.21.15.111:0/3557877225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c103880 0x7f351c103ca0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 -- 172.21.15.111:0/3557877225 >> 172.21.15.111:0/3557877225 conn(0x7f351c0fedf0 msgr2=0x7f351c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 -- 172.21.15.111:0/3557877225 shutdown_connections 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.713+0000 7f3520e00640 1 -- 172.21.15.111:0/3557877225 wait complete. 2024-04-24T22:41:44.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.714+0000 7f3520e00640 1 Processor -- start 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 -- start start 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c103880 0x7f351c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 0x7f351c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10f500 con 0x7f351c104a70 2024-04-24T22:41:44.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10f380 con 0x7f351c105430 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f3520e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10f680 con 0x7f351c103880 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f351be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f351b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c103880 0x7f351c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f351be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34080/0 (socket says 172.21.15.111:34080) 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f351be00640 1 -- 172.21.15.111:0/3992670025 learned_addr learned my addr 172.21.15.111:0/3992670025 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:44.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.715+0000 7f351aa00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 0x7f351c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 -- 172.21.15.111:0/3992670025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c103880 msgr2=0x7f351c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c103880 0x7f351c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 -- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 msgr2=0x7f351c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 0x7f351c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 -- 172.21.15.111:0/3992670025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c1a1770 con 0x7f351c105430 2024-04-24T22:41:44.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f351be00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f351400eb10 tx=0x7f351400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514004b20 con 0x7f351c105430 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3514004cc0 con 0x7f351c105430 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514010760 con 0x7f351c105430 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1a1a60 con 0x7f351c105430 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.716+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c1a1f60 con 0x7f351c105430 2024-04-24T22:41:44.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.717+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c10aba0 con 0x7f351c105430 2024-04-24T22:41:44.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.720+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3514005000 con 0x7f351c105430 2024-04-24T22:41:44.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.721+0000 7f3513e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 0x7f35000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:44.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.721+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f35140cfab0 con 0x7f351c105430 2024-04-24T22:41:44.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.721+0000 7f351b400640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 0x7f35000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:44.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.722+0000 7f351b400640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 0x7f35000a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f350c005ab0 tx=0x7f350c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:44.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.723+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f351409c9c0 con 0x7f351c105430 2024-04-24T22:41:44.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.923+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f351c062340 con 0x7f351c105430 2024-04-24T22:41:44.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.924+0000 7f3513e00640 1 -- 172.21.15.111:0/3992670025 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f351409c2c0 con 0x7f351c105430 2024-04-24T22:41:44.926 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:41:44.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 msgr2=0x7f35000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 0x7f35000a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f350c005ab0 tx=0x7f350c000bc0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 msgr2=0x7f351c1a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:44.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f351400eb10 tx=0x7f351400c2d0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 shutdown_connections 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f35000a2b90 0x7f35000a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f351c105430 0x7f351c1a1100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f351c104a70 0x7f351c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 --2- 172.21.15.111:0/3992670025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c103880 0x7f351c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.926+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 >> 172.21.15.111:0/3992670025 conn(0x7f351c0fedf0 msgr2=0x7f351c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.927+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 shutdown_connections 2024-04-24T22:41:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:44.927+0000 7f3520e00640 1 -- 172.21.15.111:0/3992670025 wait complete. 2024-04-24T22:41:44.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 -- 172.21.15.111:0/1371106057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70104b80 msgr2=0x7f6e70105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 --2- 172.21.15.111:0/1371106057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70104b80 0x7f6e70105000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009900 tx=0x7f6e5802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 -- 172.21.15.111:0/1371106057 shutdown_connections 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 --2- 172.21.15.111:0/1371106057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70105540 0x7f6e7010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 --2- 172.21.15.111:0/1371106057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70104b80 0x7f6e70105000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 --2- 172.21.15.111:0/1371106057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70103990 0x7f6e70103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 -- 172.21.15.111:0/1371106057 >> 172.21.15.111:0/1371106057 conn(0x7f6e700fef00 msgr2=0x7f6e70101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 -- 172.21.15.111:0/1371106057 shutdown_connections 2024-04-24T22:41:45.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.133+0000 7f6e76200640 1 -- 172.21.15.111:0/1371106057 wait complete. 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 Processor -- start 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 -- start start 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70103990 0x7f6e70199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 0x7f6e7019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7010f5f0 con 0x7f6e70105540 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7010f470 con 0x7f6e70103990 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.134+0000 7f6e76200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7010f770 con 0x7f6e70104b80 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50562/0 (socket says 172.21.15.111:50562) 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 -- 172.21.15.111:0/825862272 learned_addr learned my addr 172.21.15.111:0/825862272 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6ea00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 0x7f6e7019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6f400640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70103990 0x7f6e70199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 -- 172.21.15.111:0/825862272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 msgr2=0x7f6e7019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 0x7f6e7019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 -- 172.21.15.111:0/825862272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70103990 msgr2=0x7f6e70199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70103990 0x7f6e70199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 -- 172.21.15.111:0/825862272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e701a18b0 con 0x7f6e70105540 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6ea00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 0x7f6e7019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.135+0000 7f6e6fe00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e970 tx=0x7f6e6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.136+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800ccc0 con 0x7f6e70105540 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.136+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e6800ce60 con 0x7f6e70105540 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.136+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e701a1ba0 con 0x7f6e70105540 2024-04-24T22:41:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.136+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68010490 con 0x7f6e70105540 2024-04-24T22:41:45.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.136+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e701a20c0 con 0x7f6e70105540 2024-04-24T22:41:45.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.137+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7010ad80 con 0x7f6e70105540 2024-04-24T22:41:45.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.139+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6e68005020 con 0x7f6e70105540 2024-04-24T22:41:45.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.140+0000 7f6e67e00640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 0x7f6e540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.140+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6e680cf480 con 0x7f6e70105540 2024-04-24T22:41:45.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.140+0000 7f6e6f400640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 0x7f6e540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.141+0000 7f6e6f400640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 0x7f6e540a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e7019ab40 tx=0x7f6e60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:45.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.144+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e6809c390 con 0x7f6e70105540 2024-04-24T22:41:45.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:44 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1949303842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:44 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1092238906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:44 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1949303842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:44 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1092238906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:44 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1949303842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:44 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1092238906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:41:45.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.383+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e7002d170 con 0x7f6e70105540 2024-04-24T22:41:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.384+0000 7f6e67e00640 1 -- 172.21.15.111:0/825862272 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11877 (secure 0 0 0) 0x7f6e680b3e80 con 0x7f6e70105540 2024-04-24T22:41:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:45.386 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:39:35.992637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24556,24574,24587]},"id":1}]} 2024-04-24T22:41:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 msgr2=0x7f6e540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 0x7f6e540a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e7019ab40 tx=0x7f6e60009500 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 msgr2=0x7f6e701a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e970 tx=0x7f6e6800ed70 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 shutdown_connections 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f6e540a2c10 0x7f6e540a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e70105540 0x7f6e701a1210 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e70104b80 0x7f6e7019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 --2- 172.21.15.111:0/825862272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e70103990 0x7f6e70199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 >> 172.21.15.111:0/825862272 conn(0x7f6e700fef00 msgr2=0x7f6e70107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:45.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 shutdown_connections 2024-04-24T22:41:45.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.387+0000 7f6e76200640 1 -- 172.21.15.111:0/825862272 wait complete. 2024-04-24T22:41:45.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.607+0000 7f0977a00640 1 -- 172.21.15.111:0/633043369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970108f90 msgr2=0x7f0970109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 --2- 172.21.15.111:0/633043369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970108f90 0x7f0970109410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0958009900 tx=0x7f095802a410 comp rx=0 tx=0).stop 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 -- 172.21.15.111:0/633043369 shutdown_connections 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 --2- 172.21.15.111:0/633043369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970109950 0x7f09701101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 --2- 172.21.15.111:0/633043369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970108f90 0x7f0970109410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 --2- 172.21.15.111:0/633043369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970107da0 0x7f09701081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 -- 172.21.15.111:0/633043369 >> 172.21.15.111:0/633043369 conn(0x7f0970076d60 msgr2=0x7f0970079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 -- 172.21.15.111:0/633043369 shutdown_connections 2024-04-24T22:41:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.608+0000 7f0977a00640 1 -- 172.21.15.111:0/633043369 wait complete. 2024-04-24T22:41:45.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 Processor -- start 2024-04-24T22:41:45.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 -- start start 2024-04-24T22:41:45.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 0x7f097019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970108f90 0x7f097019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0970113a00 con 0x7f0970109950 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0970113880 con 0x7f0970107da0 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.609+0000 7f0977a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0970113b80 con 0x7f0970108f90 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50586/0 (socket says 172.21.15.111:50586) 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 -- 172.21.15.111:0/613028559 learned_addr learned my addr 172.21.15.111:0/613028559 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975000640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 0x7f097019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0967e00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970108f90 0x7f097019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 -- 172.21.15.111:0/613028559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970108f90 msgr2=0x7f097019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970108f90 0x7f097019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 -- 172.21.15.111:0/613028559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 msgr2=0x7f097019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 0x7f097019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 -- 172.21.15.111:0/613028559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09701a5d00 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975000640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 0x7f097019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.610+0000 7f0975a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f096800efd0 tx=0x7f096800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.611+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968003b70 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.611+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968003d10 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.611+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680109f0 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.611+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09701a5ff0 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.611+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09701a6460 con 0x7f0970109950 2024-04-24T22:41:45.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.612+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097010f140 con 0x7f0970109950 2024-04-24T22:41:45.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.614+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0968004020 con 0x7f0970109950 2024-04-24T22:41:45.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.616+0000 7f096e000640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 0x7f09540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:45.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.616+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f09680cf4b0 con 0x7f0970109950 2024-04-24T22:41:45.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.616+0000 7f0975000640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 0x7f09540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:45.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.617+0000 7f0975000640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 0x7f09540a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0970108df0 tx=0x7f0960005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:45.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.620+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f096809c3c0 con 0x7f0970109950 2024-04-24T22:41:45.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.825+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09700623e0 con 0x7f0970109950 2024-04-24T22:41:45.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.827+0000 7f096e000640 1 -- 172.21.15.111:0/613028559 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22688 (secure 0 0 0) 0x7f096809bcc0 con 0x7f0970109950 2024-04-24T22:41:45.829 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:41:45.830 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":87,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:40:28.695337+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 msgr2=0x7f09540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 0x7f09540a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0970108df0 tx=0x7f0960005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:45.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 msgr2=0x7f09701a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f096800efd0 tx=0x7f096800c6a0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 shutdown_connections 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f09540a2b90 0x7f09540a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0970109950 0x7f09701a5690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0970108f90 0x7f097019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 --2- 172.21.15.111:0/613028559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0970107da0 0x7f097019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 >> 172.21.15.111:0/613028559 conn(0x7f0970076d60 msgr2=0x7f097010c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 shutdown_connections 2024-04-24T22:41:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:45.830+0000 7f0977a00640 1 -- 172.21.15.111:0/613028559 wait complete. 2024-04-24T22:41:45.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T22:39:30.113682+0000', 'modified': '2024-04-24T22:39:35.992637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24574, 'mds_2': 24556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/964018737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 964018737}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 964018737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/1876976469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1876976469}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1876976469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/3780798137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3780798137}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3780798137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6839/2246377551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2246377551}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2246377551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24574': {'gid': 24574, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/84522869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 84522869}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 84522869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.171:6839/3821671666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 3821671666}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 3821671666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24587, 'qdb_cluster': [24556, 24574, 24587]} 2024-04-24T22:41:45.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T22:41:45.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T22:41:46.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 -- 172.21.15.111:0/753634057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 msgr2=0x7f3148100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 --2- 172.21.15.111:0/753634057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148100970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3138009900 tx=0x7f313802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 -- 172.21.15.111:0/753634057 shutdown_connections 2024-04-24T22:41:46.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 --2- 172.21.15.111:0/753634057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 0x7f3148108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 --2- 172.21.15.111:0/753634057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 0x7f3148101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 --2- 172.21.15.111:0/753634057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148100970 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 -- 172.21.15.111:0/753634057 >> 172.21.15.111:0/753634057 conn(0x7f31480fbae0 msgr2=0x7f31480fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 -- 172.21.15.111:0/753634057 shutdown_connections 2024-04-24T22:41:46.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.049+0000 7f3147200640 1 -- 172.21.15.111:0/753634057 wait complete. 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.050+0000 7f3147200640 1 Processor -- start 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.050+0000 7f3147200640 1 -- start start 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 0x7f314819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 0x7f31481a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f314810c230 con 0x7f3148102100 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f314810c0b0 con 0x7f3148100550 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3147200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f314810c3b0 con 0x7f3148101740 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 0x7f31481a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36366/0 (socket says 172.21.15.111:36366) 2024-04-24T22:41:46.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 -- 172.21.15.111:0/1933984730 learned_addr learned my addr 172.21.15.111:0/1933984730 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145400640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 0x7f314819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 -- 172.21.15.111:0/1933984730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 msgr2=0x7f314819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 0x7f314819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 -- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 msgr2=0x7f31481a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 0x7f31481a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 -- 172.21.15.111:0/1933984730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31481a17a0 con 0x7f3148100550 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.051+0000 7f3145e00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3138031440 tx=0x7f3138032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.052+0000 7f313ea00640 1 -- 172.21.15.111:0/1933984730 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313803f030 con 0x7f3148100550 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.052+0000 7f313ea00640 1 -- 172.21.15.111:0/1933984730 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3138003ce0 con 0x7f3148100550 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.052+0000 7f313ea00640 1 -- 172.21.15.111:0/1933984730 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138035070 con 0x7f3148100550 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.052+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31481a1a30 con 0x7f3148100550 2024-04-24T22:41:46.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.052+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481a1f70 con 0x7f3148100550 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 Processor -- start 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f313ea00640 1 -- 172.21.15.111:0/1933984730 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3138042030 con 0x7f3148100550 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 -- start start 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130126e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b130129960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b130021ed0 con 0x55b130121970 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b130084df0 con 0x55b1301274d0 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.055+0000 7f314fdf1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b13002d8d0 con 0x55b130122360 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50614/0 (socket says 172.21.15.111:50614) 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 -- 172.21.15.111:0/3261644308 learned_addr learned my addr 172.21.15.111:0/3261644308 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3145e00640 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130126e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3145400640 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b130129960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 -- 172.21.15.111:0/3261644308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 msgr2=0x55b130126e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130126e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 -- 172.21.15.111:0/3261644308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 msgr2=0x55b130129960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b130129960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 -- 172.21.15.111:0/3261644308 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12fd992d0 con 0x55b130121970 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3146800640 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3140009c90 tx=0x7f3140011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59821c25743112f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.056+0000 7f3137400640 1 -- 172.21.15.111:0/3261644308 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004230 con 0x55b130121970 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f3137400640 1 -- 172.21.15.111:0/3261644308 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140009e40 con 0x55b130121970 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f3137400640 1 -- 172.21.15.111:0/3261644308 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140019760 con 0x55b130121970 2024-04-24T22:41:46.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f313ea00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 0x7f31280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f313ea00640 1 -- 172.21.15.111:0/1933984730 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f31380ef800 con 0x7f3148100550 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f3145e00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 0x7f31280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 -- 172.21.15.111:0/3261644308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 msgr2=0x55b130121d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3140009c90 tx=0x7f3140011810 comp rx=0 tx=0).stop 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 -- 172.21.15.111:0/3261644308 shutdown_connections 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b130129960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130126e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 --2- 172.21.15.111:0/3261644308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130121d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 -- 172.21.15.111:0/3261644308 >> 172.21.15.111:0/3261644308 conn(0x55b13011cfe0 msgr2=0x55b13011d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 -- 172.21.15.111:0/3261644308 shutdown_connections 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f314fdf1740 1 -- 172.21.15.111:0/3261644308 wait complete. 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.057+0000 7f3145e00640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 0x7f31280a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31380ed2b0 tx=0x7f31380f5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 -- start start 2024-04-24T22:41:46.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 Processor -- start 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130121970 0x55b13012bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b1301331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b130040d20 con 0x55b1301274d0 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b130080db0 con 0x55b130122360 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b13012a480 con 0x55b130121970 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.058+0000 7f314fdf1740 10 client.0 _ll_register_callbacks cb 0x55b13007bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3146800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130121970 0x55b13012bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b1301331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36374/0 (socket says 172.21.15.111:36374) 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 -- 172.21.15.111:0/4075615497 learned_addr learned my addr 172.21.15.111:0/4075615497 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 -- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130121970 msgr2=0x55b13012bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130121970 0x55b13012bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 -- 172.21.15.111:0/4075615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 msgr2=0x55b1301331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b1301331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 -- 172.21.15.111:0/4075615497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12fe636b0 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f3145400640 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3130011cf0 tx=0x7f313000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828dc3dc4836830d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.059+0000 7f312fe00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130003970 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3130003b10 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b130092330 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3130003d10 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 10 client.25133 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b130136610 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b130092330 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f3130019070 con 0x55b130122360 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 10 client.25133 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 10 client.25133 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 20 client.25133 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b130138c40 0x55b13013a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 4 client.25133 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.060+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 --> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13008c680 con 0x55b130138c40 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.061+0000 7f3146800640 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b130138c40 0x55b13013a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.061+0000 7f3146800640 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b130138c40 0x55b13013a0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eecb00fb6832c247 server_cookie=9a4f3398a24e9f0e in_seq=0 out_seq=0 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.061+0000 7f312fe00640 10 client.25133 ms_handle_connect on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f312fe00640 1 -- 172.21.15.111:0/4075615497 <== mds.2 v2:172.21.15.186:6838/2246377551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b13008c680 con 0x55b130138c40 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f312fe00640 10 client.25133 handle_command_reply: tid=0 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 client.25133 shutdown 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 msgr2=0x55b13012c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3130011cf0 tx=0x7f313000f790 comp rx=0 tx=0).stop 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 shutdown_connections 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b130138c40 0x55b13013a0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b1301331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130122360 0x55b13012c1b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f312fe00640 0 client.25133 ms_handle_reset on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 --2- 172.21.15.111:0/4075615497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130121970 0x55b13012bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 >> 172.21.15.111:0/4075615497 conn(0x55b13011cfe0 msgr2=0x55b13011c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 shutdown_connections 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.062+0000 7f314fdf1740 1 -- 172.21.15.111:0/4075615497 wait complete. 2024-04-24T22:41:46.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.063+0000 7f314fdf1740 20 client.25133 trim_cache size 0 max 16384 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 Processor -- start 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 -- start start 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130133170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b1301298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1301383f0 con 0x55b130121970 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b130138ac0 con 0x55b1301274d0 2024-04-24T22:41:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f314fdf1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b130139190 con 0x55b130122360 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50624/0 (socket says 172.21.15.111:50624) 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f3146800640 1 -- 172.21.15.111:0/1045694646 learned_addr learned my addr 172.21.15.111:0/1045694646 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.082+0000 7f3145400640 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130133170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b1301298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 -- 172.21.15.111:0/1045694646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 msgr2=0x55b130133170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130133170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 -- 172.21.15.111:0/1045694646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 msgr2=0x55b1301298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b1301298c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 -- 172.21.15.111:0/1045694646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13011dd50 con 0x55b130121970 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3146800640 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31400097d0 tx=0x7f3140009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c2d4a949253ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3136a00640 1 -- 172.21.15.111:0/1045694646 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400041a0 con 0x55b130121970 2024-04-24T22:41:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3136a00640 1 -- 172.21.15.111:0/1045694646 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140003e70 con 0x55b130121970 2024-04-24T22:41:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.083+0000 7f3136a00640 1 -- 172.21.15.111:0/1045694646 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004680 con 0x55b130121970 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- 172.21.15.111:0/1045694646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 msgr2=0x55b130117ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31400097d0 tx=0x7f3140009980 comp rx=0 tx=0).stop 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- 172.21.15.111:0/1045694646 shutdown_connections 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b1301274d0 0x55b1301298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b130133170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 --2- 172.21.15.111:0/1045694646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b130121970 0x55b130117ea0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- 172.21.15.111:0/1045694646 >> 172.21.15.111:0/1045694646 conn(0x55b13011cfe0 msgr2=0x55b13011d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- 172.21.15.111:0/1045694646 shutdown_connections 2024-04-24T22:41:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- 172.21.15.111:0/1045694646 wait complete. 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.084+0000 7f314fdf1740 1 -- start start 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 Processor -- start 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130121970 0x55b13012e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b13012ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b130139f30 con 0x55b1301274d0 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b130139860 con 0x55b130121970 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b13013a600 con 0x55b130122360 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f314fdf1740 10 client.0 _ll_register_callbacks cb 0x7f3130009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f3146800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50638/0 (socket says 172.21.15.111:50638) 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f3146800640 1 -- 172.21.15.111:0/2904252574 learned_addr learned my addr 172.21.15.111:0/2904252574 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f3145400640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b13012ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.085+0000 7f3146800640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130121970 0x55b13012e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 -- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 msgr2=0x55b13012ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b13012ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 -- 172.21.15.111:0/2904252574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130121970 msgr2=0x55b13012e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130121970 0x55b13012e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 -- 172.21.15.111:0/2904252574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b130132160 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f3146800640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3140003c50 tx=0x7f3140005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c02b60b7c2b581 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f312fe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140003670 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3140005030 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140003e70 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3140005270 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f314fdf1740 10 client.15051 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.086+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b130133460 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140003e70 con 0x55b1301274d0 2024-04-24T22:41:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f314003f060 con 0x55b1301274d0 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 10 client.15051 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 20 client.15051 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b1301a2e00 0x55b1301a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 4 client.15051 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 --> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] -- command(tid 0: {"prefix": "status"}) -- 0x55b13000aa40 con 0x55b1301a2e00 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.087+0000 7f3145400640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b1301a2e00 0x55b1301a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f312fe00640 10 client.15051 ms_handle_connect on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f3145400640 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b1301a2e00 0x55b1301a1b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d605a3ecc08b2ff server_cookie=3f4797eeed71fc4b in_seq=0 out_seq=0 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f312fe00640 1 -- 172.21.15.111:0/2904252574 <== mds.2 v2:172.21.15.186:6838/2246377551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55b13000aa40 con 0x55b1301a2e00 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f312fe00640 10 client.15051 handle_command_reply: tid=0 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 client.15051 shutdown 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 msgr2=0x55b130131b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3140003c50 tx=0x7f3140005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 shutdown_connections 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:6838/2246377551,v1:172.21.15.186:6839/2246377551] conn(0x55b1301a2e00 0x55b1301a1b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1301274d0 0x55b130131b70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b130122360 0x55b13012ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 --2- 172.21.15.111:0/2904252574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b130121970 0x55b13012e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 >> 172.21.15.111:0/2904252574 conn(0x55b13011cfe0 msgr2=0x55b13011d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f312fe00640 0 client.15051 ms_handle_reset on v2:172.21.15.186:6838/2246377551 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 shutdown_connections 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.088+0000 7f314fdf1740 1 -- 172.21.15.111:0/2904252574 wait complete. 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f314fdf1740 20 client.15051 trim_cache size 0 max 16384 2024-04-24T22:41:46.095 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 2, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24556, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 131.111234838, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 151.37685868 2024-04-24T22:41:46.096 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 msgr2=0x7f31280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 0x7f31280a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31380ed2b0 tx=0x7f31380f5530 comp rx=0 tx=0).stop 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 msgr2=0x7f3148199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3138031440 tx=0x7f3138032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 shutdown_connections 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f31280a2c10 0x7f31280a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3148102100 0x7f31481a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3148101740 0x7f314819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 --2- 172.21.15.111:0/1933984730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3148100550 0x7f3148199b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 >> 172.21.15.111:0/1933984730 conn(0x7f31480fbae0 msgr2=0x7f31480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 shutdown_connections 2024-04-24T22:41:46.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.089+0000 7f3147200640 1 -- 172.21.15.111:0/1933984730 wait complete. 2024-04-24T22:41:46.105 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:45 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3992670025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:46.106 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:45 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/825862272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.106 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:45 smithi111 ceph-mon[27746]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:46.106 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:45 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/613028559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-24T22:41:46.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:45 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3992670025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:46.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:45 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/825862272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:45 smithi171 ceph-mon[32337]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:46.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:45 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/613028559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:45 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3992670025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:41:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:45 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/825862272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:45 smithi186 ceph-mon[32753]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:41:46.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:45 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/613028559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:46.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 -- 172.21.15.111:0/1820672293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00102980 msgr2=0x7f8e00102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 --2- 172.21.15.111:0/1820672293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00102980 0x7f8e00102e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009900 tx=0x7f8dec02a410 comp rx=0 tx=0).stop 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 -- 172.21.15.111:0/1820672293 shutdown_connections 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 --2- 172.21.15.111:0/1820672293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00103340 0x7f8e00109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 --2- 172.21.15.111:0/1820672293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00102980 0x7f8e00102e00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.301+0000 7f8e07600640 1 --2- 172.21.15.111:0/1820672293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 0x7f8e00101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.302+0000 7f8e07600640 1 -- 172.21.15.111:0/1820672293 >> 172.21.15.111:0/1820672293 conn(0x7f8e000fcd20 msgr2=0x7f8e000ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.302+0000 7f8e07600640 1 -- 172.21.15.111:0/1820672293 shutdown_connections 2024-04-24T22:41:46.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.302+0000 7f8e07600640 1 -- 172.21.15.111:0/1820672293 wait complete. 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 Processor -- start 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 -- start start 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 0x7f8e00199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 0x7f8e001a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0010d410 con 0x7f8e00102980 2024-04-24T22:41:46.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0010d290 con 0x7f8e00103340 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e07600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0010d590 con 0x7f8e00101790 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8dffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8dffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50666/0 (socket says 172.21.15.111:50666) 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8dffe00640 1 -- 172.21.15.111:0/4263529065 learned_addr learned my addr 172.21.15.111:0/4263529065 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e04c00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 0x7f8e00199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.303+0000 7f8e05600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 0x7f8e001a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 -- 172.21.15.111:0/4263529065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 msgr2=0x7f8e00199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 0x7f8e00199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 -- 172.21.15.111:0/4263529065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 msgr2=0x7f8e001a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 0x7f8e001a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 -- 172.21.15.111:0/4263529065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e001a1950 con 0x7f8e00102980 2024-04-24T22:41:46.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8e05600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 0x7f8e001a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dffe00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009230 tx=0x7f8dec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dfd600640 1 -- 172.21.15.111:0/4263529065 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec03f030 con 0x7f8e00102980 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dfd600640 1 -- 172.21.15.111:0/4263529065 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dec003ce0 con 0x7f8e00102980 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e001a1be0 con 0x7f8e00102980 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.304+0000 7f8dfd600640 1 -- 172.21.15.111:0/4263529065 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec035070 con 0x7f8e00102980 2024-04-24T22:41:46.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.305+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e001a20d0 con 0x7f8e00102980 2024-04-24T22:41:46.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 Processor -- start 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 -- start start 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8dfd600640 1 -- 172.21.15.111:0/4263529065 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8dec042030 con 0x7f8e00102980 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a30785b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a3078d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a30685ed0 con 0x555a30786150 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555a306e8df0 con 0x555a3078b2c0 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a306918d0 con 0x555a30785760 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.309+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50674/0 (socket says 172.21.15.111:50674) 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e04c00640 1 -- 172.21.15.111:0/2031046246 learned_addr learned my addr 172.21.15.111:0/2031046246 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a30785b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8dffe00640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a3078d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 -- 172.21.15.111:0/2031046246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 msgr2=0x555a3078d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a3078d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 -- 172.21.15.111:0/2031046246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 msgr2=0x555a3078ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 -- 172.21.15.111:0/2031046246 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a303fd2d0 con 0x555a30785760 2024-04-24T22:41:46.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e04c00640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:46.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.310+0000 7f8e05600640 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a30785b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df8011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5dfc0a192442186 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8df1c00640 1 -- 172.21.15.111:0/2031046246 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8004d20 con 0x555a30785760 2024-04-24T22:41:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8dfd600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 0x7f8de80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8df1c00640 1 -- 172.21.15.111:0/2031046246 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df800f9a0 con 0x555a30785760 2024-04-24T22:41:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8df1c00640 1 -- 172.21.15.111:0/2031046246 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8004490 con 0x555a30785760 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8dfd600640 1 -- 172.21.15.111:0/4263529065 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8dec0ef590 con 0x7f8e00102980 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.311+0000 7f8e04c00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 0x7f8de80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2031046246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 msgr2=0x555a30785b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a30785b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df8011bf0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e04c00640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 0x7f8de80a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x555a30786550 tx=0x7f8df4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2031046246 shutdown_connections 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a3078d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a3078ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2031046246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a30785b80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2031046246 >> 172.21.15.111:0/2031046246 conn(0x555a30780dd0 msgr2=0x555a307811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2031046246 shutdown_connections 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2031046246 wait complete. 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 -- start start 2024-04-24T22:41:46.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.312+0000 7f8e0a3ee740 1 Processor -- start 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a3078ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30796fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a306a4d20 con 0x555a30785760 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555a306e4db0 con 0x555a3078b2c0 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a3078e270 con 0x555a30786150 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e0a3ee740 10 client.0 _ll_register_callbacks cb 0x555a306dff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e05600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30796fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8dffe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a3078ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50690/0 (socket says 172.21.15.111:50690) 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e04c00640 1 -- 172.21.15.111:0/2317211579 learned_addr learned my addr 172.21.15.111:0/2317211579 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.313+0000 7f8e05600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30796fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36426/0 (socket says 172.21.15.111:36426) 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 -- 172.21.15.111:0/2317211579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 msgr2=0x555a30796fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30796fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 -- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 msgr2=0x555a3078fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 -- 172.21.15.111:0/2317211579 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a304c76b0 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8e04c00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8dffe00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a3078ffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0f2fd0 tx=0x7f8dec005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5977a8aa181a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8de6000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec049040 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8dec03bc10 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df800f9a0 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8dec0b8060 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8e0a3ee740 10 client.25162 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.314+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555a3079a350 con 0x555a30786150 2024-04-24T22:41:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df800f9a0 con 0x555a30786150 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8dec0d3030 con 0x555a30786150 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 10 client.25162 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 10 client.25162 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 20 client.25162 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3079ca00 0x555a3079de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 4 client.25162 mds_command: new command op to 24574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 --> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a306f6330 con 0x555a3079ca00 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.315+0000 7f8e04c00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3079ca00 0x555a3079de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.316+0000 7f8e04c00640 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3079ca00 0x555a3079de70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ff8e25599c4965 server_cookie=cd1430de1cdb1d05 in_seq=0 out_seq=0 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.316+0000 7f8de6000640 10 client.25162 ms_handle_connect on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8de6000640 1 -- 172.21.15.111:0/2317211579 <== mds.1 v2:172.21.15.111:6838/84522869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a306f6330 con 0x555a3079ca00 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8de6000640 10 client.25162 handle_command_reply: tid=0 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 client.25162 shutdown 2024-04-24T22:41:46.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 msgr2=0x555a3078ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a3078ffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0f2fd0 tx=0x7f8dec005b50 comp rx=0 tx=0).stop 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 shutdown_connections 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30796fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a3078ffa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3079ca00 0x555a3079de70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/2317211579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a3078fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8de6000640 0 client.25162 ms_handle_reset on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 >> 172.21.15.111:0/2317211579 conn(0x555a30780dd0 msgr2=0x555a3077ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 shutdown_connections 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/2317211579 wait complete. 2024-04-24T22:41:46.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.317+0000 7f8e0a3ee740 20 client.25162 trim_cache size 0 max 16384 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 Processor -- start 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 -- start start 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a30796fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30787a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a3079c1b0 con 0x555a30785760 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555a3079c880 con 0x555a3078b2c0 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.337+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a3079cf50 con 0x555a30786150 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50696/0 (socket says 172.21.15.111:50696) 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 -- 172.21.15.111:0/811289350 learned_addr learned my addr 172.21.15.111:0/811289350 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e05600640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30787a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8dffe00640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a30796fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 -- 172.21.15.111:0/811289350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 msgr2=0x555a30796fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a30796fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 -- 172.21.15.111:0/811289350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 msgr2=0x555a30787a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30787a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e04c00640 1 -- 172.21.15.111:0/811289350 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3077f410 con 0x555a30785760 2024-04-24T22:41:46.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8dffe00640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a30796fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:46.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.338+0000 7f8e05600640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30787a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:46.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8e04c00640 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555a3079e5b0 tx=0x7f8df4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7975218f0c09a52a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8df1200640 1 -- 172.21.15.111:0/811289350 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df401ec80 con 0x555a30785760 2024-04-24T22:41:46.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8df1200640 1 -- 172.21.15.111:0/811289350 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df401ee20 con 0x555a30785760 2024-04-24T22:41:46.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8df1200640 1 -- 172.21.15.111:0/811289350 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4037430 con 0x555a30785760 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/811289350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 msgr2=0x555a30787550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.339+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555a3079e5b0 tx=0x7f8df4017000 comp rx=0 tx=0).stop 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/811289350 shutdown_connections 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30787a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30786150 0x555a30796fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/811289350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30785760 0x555a30787550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/811289350 >> 172.21.15.111:0/811289350 conn(0x555a30780dd0 msgr2=0x555a307811b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/811289350 shutdown_connections 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/811289350 wait complete. 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 -- start start 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.340+0000 7f8e0a3ee740 1 Processor -- start 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a307926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30795bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a3079dcf0 con 0x555a30786150 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555a3079d620 con 0x555a3078b2c0 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a3079e3c0 con 0x555a30785760 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e0a3ee740 10 client.0 _ll_register_callbacks cb 0x7f8dec031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50710/0 (socket says 172.21.15.111:50710) 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 -- 172.21.15.111:0/309443513 learned_addr learned my addr 172.21.15.111:0/309443513 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e05600640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30795bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8e04c00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a307926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 -- 172.21.15.111:0/309443513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 msgr2=0x555a307926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a307926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 -- 172.21.15.111:0/309443513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 msgr2=0x555a30795bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30795bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 -- 172.21.15.111:0/309443513 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a307961c0 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8dffe00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8dec031b40 tx=0x7f8dec0b8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956aabb2c6db588 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.341+0000 7f8de6000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec049070 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8dec0ef2d0 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df401ee20 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8dec0044e0 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 10 client.15069 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555a3080b230 con 0x555a30786150 2024-04-24T22:41:46.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df401ee20 con 0x555a30786150 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8dec031b90 con 0x555a30786150 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 10 client.15069 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 10 client.15069 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 20 client.15069 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3080fbf0 0x555a3080c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 4 client.15069 mds_command: new command op to 24574 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.342+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 --> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] -- command(tid 0: {"prefix": "status"}) -- 0x555a306f0680 con 0x555a3080fbf0 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8e04c00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3080fbf0 0x555a3080c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8e04c00640 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3080fbf0 0x555a3080c700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b00f373cadb18e2 server_cookie=143060e297903f55 in_seq=0 out_seq=0 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8de6000640 10 client.15069 ms_handle_connect on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8de6000640 1 -- 172.21.15.111:0/309443513 <== mds.1 v2:172.21.15.111:6838/84522869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x555a306f0680 con 0x555a3080fbf0 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8de6000640 10 client.15069 handle_command_reply: tid=0 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.343+0000 7f8e0a3ee740 1 client.15069 shutdown 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 msgr2=0x555a30792c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8dec031b40 tx=0x7f8dec0b8640 comp rx=0 tx=0).stop 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 shutdown_connections 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555a3078b2c0 0x555a30795bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:6838/84522869,v1:172.21.15.111:6839/84522869] conn(0x555a3080fbf0 0x555a3080c700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a30786150 0x555a30792c00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 --2- 172.21.15.111:0/309443513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a30785760 0x555a307926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 >> 172.21.15.111:0/309443513 conn(0x555a30780dd0 msgr2=0x555a3079b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8de6000640 0 client.15069 ms_handle_reset on v2:172.21.15.111:6838/84522869 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 shutdown_connections 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 1 -- 172.21.15.111:0/309443513 wait complete. 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.344+0000 7f8e0a3ee740 20 client.15069 trim_cache size 0 max 16384 2024-04-24T22:41:46.352 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 1, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24574, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 132.38095529, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 147.663369773 2024-04-24T22:41:46.353 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 msgr2=0x7f8de80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 0x7f8de80a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x555a30786550 tx=0x7f8df4005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 msgr2=0x7f8e0019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009230 tx=0x7f8dec0042f0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 shutdown_connections 2024-04-24T22:41:46.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f8de80a2b90 0x7f8de80a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e00103340 0x7f8e001a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00102980 0x7f8e0019a280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 --2- 172.21.15.111:0/4263529065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e00101790 0x7f8e00199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 >> 172.21.15.111:0/4263529065 conn(0x7f8e000fcd20 msgr2=0x7f8e00073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 shutdown_connections 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.345+0000 7f8e07600640 1 -- 172.21.15.111:0/4263529065 wait complete. 2024-04-24T22:41:46.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 -- 172.21.15.111:0/2500671659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0102970 msgr2=0x7f68d0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 --2- 172.21.15.111:0/2500671659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0102970 0x7f68d0102df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009900 tx=0x7f68bc02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 -- 172.21.15.111:0/2500671659 shutdown_connections 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 --2- 172.21.15.111:0/2500671659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0103330 0x7f68d0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 --2- 172.21.15.111:0/2500671659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0102970 0x7f68d0102df0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 --2- 172.21.15.111:0/2500671659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0101780 0x7f68d0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 -- 172.21.15.111:0/2500671659 >> 172.21.15.111:0/2500671659 conn(0x7f68d00fccf0 msgr2=0x7f68d00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 -- 172.21.15.111:0/2500671659 shutdown_connections 2024-04-24T22:41:46.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.559+0000 7f68d7e00640 1 -- 172.21.15.111:0/2500671659 wait complete. 2024-04-24T22:41:46.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.560+0000 7f68d7e00640 1 Processor -- start 2024-04-24T22:41:46.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.560+0000 7f68d7e00640 1 -- start start 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0102970 0x7f68d0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0103330 0x7f68d019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010d450 con 0x7f68d0101780 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010d2d0 con 0x7f68d0103330 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010d5d0 con 0x7f68d0102970 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50724/0 (socket says 172.21.15.111:50724) 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d5400640 1 -- 172.21.15.111:0/3674270241 learned_addr learned my addr 172.21.15.111:0/3674270241 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0103330 0x7f68d019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.561+0000 7f68d4a00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0102970 0x7f68d0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 -- 172.21.15.111:0/3674270241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0102970 msgr2=0x7f68d0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0102970 0x7f68d0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 -- 172.21.15.111:0/3674270241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0103330 msgr2=0x7f68d019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0103330 0x7f68d019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 -- 172.21.15.111:0/3674270241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d019f6e0 con 0x7f68d0101780 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d5400640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68c400e9c0 tx=0x7f68c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68ce000640 1 -- 172.21.15.111:0/3674270241 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c400cd80 con 0x7f68d0101780 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68ce000640 1 -- 172.21.15.111:0/3674270241 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c40035f0 con 0x7f68d0101780 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d019f9d0 con 0x7f68d0101780 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68ce000640 1 -- 172.21.15.111:0/3674270241 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4010690 con 0x7f68d0101780 2024-04-24T22:41:46.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.562+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d019fef0 con 0x7f68d0101780 2024-04-24T22:41:46.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.565+0000 7f68dab99740 1 Processor -- start 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 -- start start 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68ce000640 1 -- 172.21.15.111:0/3674270241 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f68c40037a0 con 0x7f68d0101780 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced83b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626ced88c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626cec83ed0 con 0x5626ced84150 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626cece6df0 con 0x5626ced892c0 2024-04-24T22:41:46.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68dab99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626cec8f8d0 con 0x5626ced83760 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68d4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68d5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626ced88c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68d4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36494/0 (socket says 172.21.15.111:36494) 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced83b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.566+0000 7f68d4a00640 1 -- 172.21.15.111:0/1442079248 learned_addr learned my addr 172.21.15.111:0/1442079248 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 -- 172.21.15.111:0/1442079248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 msgr2=0x5626ced8b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 -- 172.21.15.111:0/1442079248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 msgr2=0x5626ced88c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626ced88c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 -- 172.21.15.111:0/1442079248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ce9fb2d0 con 0x5626ced83760 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d4a00640 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68d5e00640 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced83b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f30 tx=0x7f68c8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7bf35a675cffca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68c2600640 1 -- 172.21.15.111:0/1442079248 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8016070 con 0x5626ced83760 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68ce000640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 0x7f68b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68c2600640 1 -- 172.21.15.111:0/1442079248 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8004d20 con 0x5626ced83760 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.567+0000 7f68c2600640 1 -- 172.21.15.111:0/1442079248 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8012870 con 0x5626ced83760 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68d5400640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 0x7f68b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68ce000640 1 -- 172.21.15.111:0/3674270241 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f68c40cf4c0 con 0x7f68d0101780 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 -- 172.21.15.111:0/1442079248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 msgr2=0x5626ced83b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced83b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f30 tx=0x7f68c8010800 comp rx=0 tx=0).stop 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68d5400640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 0x7f68b80a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f68c40cd340 tx=0x7f68c40d5060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 -- 172.21.15.111:0/1442079248 shutdown_connections 2024-04-24T22:41:46.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626ced8b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626ced88c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 --2- 172.21.15.111:0/1442079248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced83b80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 -- 172.21.15.111:0/1442079248 >> 172.21.15.111:0/1442079248 conn(0x5626ced7edd0 msgr2=0x5626ced7f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 -- 172.21.15.111:0/1442079248 shutdown_connections 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.568+0000 7f68dab99740 1 -- 172.21.15.111:0/1442079248 wait complete. 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 -- start start 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 Processor -- start 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced8da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced94fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626ceca2d20 con 0x5626ced892c0 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626cece2db0 con 0x5626ced84150 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626ced8c270 con 0x5626ced83760 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.569+0000 7f68dab99740 10 client.0 _ll_register_callbacks cb 0x5626cecddf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36496/0 (socket says 172.21.15.111:36496) 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 -- 172.21.15.111:0/3326441991 learned_addr learned my addr 172.21.15.111:0/3326441991 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced8da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced94fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 -- 172.21.15.111:0/3326441991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 msgr2=0x5626ced8da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced8da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 -- 172.21.15.111:0/3326441991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 msgr2=0x5626ced94fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced94fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 -- 172.21.15.111:0/3326441991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ceac56b0 con 0x5626ced84150 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced8da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:46.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68d4a00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0314e0 tx=0x7f68bc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3bffd8c946697a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.570+0000 7f68b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc035040 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c8004d20 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f68bc0318b0 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f68bc031ab0 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68dab99740 10 client.25166 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5626ced98390 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.571+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8004d20 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f68bc0035f0 con 0x5626ced84150 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 10 client.25166 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 10 client.25166 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 20 client.25166 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626ced9a930 0x5626ced9bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 4 client.25166 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626cecf4330 con 0x5626ced9a930 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.572+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626ced9a930 0x5626ced9bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.573+0000 7f68d5e00640 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626ced9a930 0x5626ced9bda0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac1f472dc2ae2b6 server_cookie=4ea64f49d6a720aa in_seq=0 out_seq=0 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.573+0000 7f68b6a00640 10 client.25166 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68b6a00640 1 -- 172.21.15.111:0/3326441991 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626cecf4330 con 0x5626ced9a930 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68b6a00640 10 client.25166 handle_command_reply: tid=0 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 client.25166 shutdown 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 msgr2=0x5626ced8df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0314e0 tx=0x7f68bc005270 comp rx=0 tx=0).stop 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 shutdown_connections 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626ced9a930 0x5626ced9bda0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced94fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced84150 0x5626ced8df70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 --2- 172.21.15.111:0/3326441991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626ced8da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68b6a00640 0 client.25166 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:46.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 >> 172.21.15.111:0/3326441991 conn(0x5626ced7edd0 msgr2=0x5626ced7df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 shutdown_connections 2024-04-24T22:41:46.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 1 -- 172.21.15.111:0/3326441991 wait complete. 2024-04-24T22:41:46.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.574+0000 7f68dab99740 20 client.25166 trim_cache size 0 max 16384 2024-04-24T22:41:46.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.597+0000 7f68dab99740 1 Processor -- start 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.597+0000 7f68dab99740 1 -- start start 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 0x5626ced85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced7d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9a0e0 con 0x5626ced892c0 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9a7b0 con 0x5626ced83760 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68dab99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9ae80 con 0x5626ced84150 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced7d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36510/0 (socket says 172.21.15.111:36510) 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.598+0000 7f68d5e00640 1 -- 172.21.15.111:0/227261448 learned_addr learned my addr 172.21.15.111:0/227261448 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d4a00640 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 0x5626ced85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 -- 172.21.15.111:0/227261448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 msgr2=0x5626ced85ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 0x5626ced85ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 -- 172.21.15.111:0/227261448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 msgr2=0x5626ced7d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced7d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 -- 172.21.15.111:0/227261448 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ced7e970 con 0x5626ced83760 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d4a00640 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 0x5626ced85ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:46.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68d5e00640 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009a80 tx=0x7f68c8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46271dcc4bc6b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68c1c00640 1 -- 172.21.15.111:0/227261448 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8016040 con 0x5626ced83760 2024-04-24T22:41:46.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68c1c00640 1 -- 172.21.15.111:0/227261448 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8003e50 con 0x5626ced83760 2024-04-24T22:41:46.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.599+0000 7f68c1c00640 1 -- 172.21.15.111:0/227261448 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8004030 con 0x5626ced83760 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 -- 172.21.15.111:0/227261448 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 msgr2=0x5626ced85570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009a80 tx=0x7f68c8009600 comp rx=0 tx=0).stop 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 -- 172.21.15.111:0/227261448 shutdown_connections 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced892c0 0x5626ced7d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced84150 0x5626ced85ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 --2- 172.21.15.111:0/227261448 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced83760 0x5626ced85570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 -- 172.21.15.111:0/227261448 >> 172.21.15.111:0/227261448 conn(0x5626ced7edd0 msgr2=0x5626ced7f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 -- 172.21.15.111:0/227261448 shutdown_connections 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.600+0000 7f68dab99740 1 -- 172.21.15.111:0/227261448 wait complete. 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 -- start start 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 Processor -- start 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626cedfe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626cee00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9bc20 con 0x5626ced84150 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9b550 con 0x5626ced892c0 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626ced9c2f0 con 0x5626ced83760 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68dab99740 10 client.0 _ll_register_callbacks cb 0x7f68bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68d4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626cee00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.601+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36520/0 (socket says 172.21.15.111:36520) 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d5e00640 1 -- 172.21.15.111:0/2276494328 learned_addr learned my addr 172.21.15.111:0/2276494328 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d5e00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626cedfe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 -- 172.21.15.111:0/2276494328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 msgr2=0x5626cedfe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626cedfe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 -- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 msgr2=0x5626cee034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 -- 172.21.15.111:0/2276494328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cee03ce0 con 0x5626ced84150 2024-04-24T22:41:46.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d5e00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68d4a00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626cee00e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f68bc003a90 tx=0x7f68bc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b20ba8e170385a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.602+0000 7f68b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc035070 con 0x5626ced84150 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c8003e50 con 0x5626ced84150 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f68bc0048c0 con 0x5626ced84150 2024-04-24T22:41:46.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f68bc05f050 con 0x5626ced84150 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68dab99740 10 client.15087 fetch_fsmap learned FSMap version 23 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5626cee04fb0 con 0x5626ced84150 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8003e50 con 0x5626ced84150 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.603+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f68bc045640 con 0x5626ced84150 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 10 client.15087 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 20 client.15087 populate_metadata read hostname 'smithi111' 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626cee09b90 0x5626cee088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 4 client.15087 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "status"}) -- 0x5626cecee680 con 0x5626cee09b90 2024-04-24T22:41:46.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68d5e00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626cee09b90 0x5626cee088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68d5e00640 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626cee09b90 0x5626cee088a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56e202dcc50a52f server_cookie=1301dedef396de9c in_seq=0 out_seq=0 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.604+0000 7f68b6a00640 10 client.15087 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68b6a00640 1 -- 172.21.15.111:0/2276494328 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5626cecee680 con 0x5626cee09b90 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68b6a00640 10 client.15087 handle_command_reply: tid=0 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 client.15087 shutdown 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 msgr2=0x5626cee00e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626cee00e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f68bc003a90 tx=0x7f68bc031400 comp rx=0 tx=0).stop 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 shutdown_connections 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5626cee09b90 0x5626cee088a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626ced892c0 0x5626cee034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68b6a00640 0 client.15087 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626ced84150 0x5626cee00e20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 --2- 172.21.15.111:0/2276494328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626ced83760 0x5626cedfe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 >> 172.21.15.111:0/2276494328 conn(0x5626ced7edd0 msgr2=0x5626ced85570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 shutdown_connections 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 1 -- 172.21.15.111:0/2276494328 wait complete. 2024-04-24T22:41:46.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.605+0000 7f68dab99740 20 client.15087 trim_cache size 0 max 16384 2024-04-24T22:41:46.614 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:41:46.614 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "0d3ba9b8-028a-11ef-bc93-c7b262605968", 2024-04-24T22:41:46.614 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-04-24T22:41:46.614 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24587, 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 135.93488248599999, 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 87, 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 85, 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 141.13925568600001 2024-04-24T22:41:46.615 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:41:46.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 msgr2=0x7f68b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 0x7f68b80a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f68c40cd340 tx=0x7f68c40d5060 comp rx=0 tx=0).stop 2024-04-24T22:41:46.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 msgr2=0x7f68d0197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68c400e9c0 tx=0x7f68c400ee80 comp rx=0 tx=0).stop 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 shutdown_connections 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f68b80a2b90 0x7f68b80a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68d0103330 0x7f68d019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d0102970 0x7f68d0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 --2- 172.21.15.111:0/3674270241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68d0101780 0x7f68d0197ac0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 >> 172.21.15.111:0/3674270241 conn(0x7f68d00fccf0 msgr2=0x7f68d0105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 shutdown_connections 2024-04-24T22:41:46.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.606+0000 7f68d7e00640 1 -- 172.21.15.111:0/3674270241 wait complete. 2024-04-24T22:41:46.627 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T22:41:46.637 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T22:41:46.637 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T22:41:46.638 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T22:41:46.638 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-24T22:41:46.639 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T22:41:46.639 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:46.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 -- 172.21.15.111:0/289489436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 msgr2=0x7faee0076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 --2- 172.21.15.111:0/289489436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee0076520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faed0009900 tx=0x7faed002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 -- 172.21.15.111:0/289489436 shutdown_connections 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 --2- 172.21.15.111:0/289489436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077ce0 0x7faee0078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 --2- 172.21.15.111:0/289489436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077340 0x7faee00777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 --2- 172.21.15.111:0/289489436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee0076520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 -- 172.21.15.111:0/289489436 >> 172.21.15.111:0/289489436 conn(0x7faee00fef30 msgr2=0x7faee0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 -- 172.21.15.111:0/289489436 shutdown_connections 2024-04-24T22:41:46.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.825+0000 7faee4c00640 1 -- 172.21.15.111:0/289489436 wait complete. 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 Processor -- start 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 -- start start 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077340 0x7faee019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faee0113a30 con 0x7faee0077340 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faee01138b0 con 0x7faee0076100 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.826+0000 7faee4c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faee0113bb0 con 0x7faee0077ce0 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faedde00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faed5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077340 0x7faee019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faedde00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36538/0 (socket says 172.21.15.111:36538) 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:46892/0 (socket says 172.21.15.111:46892) 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faedde00640 1 -- 172.21.15.111:0/3876708630 learned_addr learned my addr 172.21.15.111:0/3876708630 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 -- 172.21.15.111:0/3876708630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 msgr2=0x7faee019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 -- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077340 msgr2=0x7faee019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077340 0x7faee019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 -- 172.21.15.111:0/3876708630 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee01a5c80 con 0x7faee0077ce0 2024-04-24T22:41:46.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.827+0000 7faede800640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faed8009fd0 tx=0x7faed800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.828+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800ce20 con 0x7faee0077ce0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.828+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed8005020 con 0x7faee0077ce0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.828+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee01a5f70 con 0x7faee0077ce0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.828+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8010720 con 0x7faee0077ce0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.828+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee01a6430 con 0x7faee0077ce0 2024-04-24T22:41:46.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.829+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee0110310 con 0x7faee0077ce0 2024-04-24T22:41:46.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.831+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faed8005540 con 0x7faee0077ce0 2024-04-24T22:41:46.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.832+0000 7faed7400640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 0x7faec40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.832+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7faed80cf7b0 con 0x7faee0077ce0 2024-04-24T22:41:46.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.833+0000 7faedde00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 0x7faec40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.833+0000 7faedde00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 0x7faec40a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faed0031440 tx=0x7faed00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:46.834+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faed809c6c0 con 0x7faee0077ce0 2024-04-24T22:41:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.087+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7faee002d3c0 con 0x7faee0077ce0 2024-04-24T22:41:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.169+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7faed8098630 con 0x7faee0077ce0 2024-04-24T22:41:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.170+0000 7faed7400640 1 -- 172.21.15.111:0/3876708630 <== mon.1 v2:172.21.15.186:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7faed80d10f0 con 0x7faee0077ce0 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 msgr2=0x7faec40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 0x7faec40a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faed0031440 tx=0x7faed00315b0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 msgr2=0x7faee01a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faed8009fd0 tx=0x7faed800ef10 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 shutdown_connections 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7faec40a2cb0 0x7faec40a5170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee0077ce0 0x7faee01a5610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee0077340 0x7faee019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 --2- 172.21.15.111:0/3876708630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faee0076100 0x7faee019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 >> 172.21.15.111:0/3876708630 conn(0x7faee00fef30 msgr2=0x7faee0100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 shutdown_connections 2024-04-24T22:41:47.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.175+0000 7faee4c00640 1 -- 172.21.15.111:0/3876708630 wait complete. 2024-04-24T22:41:47.196 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-24T22:41:47.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 -- 172.21.15.111:0/2495112187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9641022a0 msgr2=0x7fc964104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 --2- 172.21.15.111:0/2495112187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9641022a0 0x7fc964104690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc95402a410 comp rx=0 tx=0).stop 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 -- 172.21.15.111:0/2495112187 shutdown_connections 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 --2- 172.21.15.111:0/2495112187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc964107500 0x7fc9641098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 --2- 172.21.15.111:0/2495112187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc964106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 --2- 172.21.15.111:0/2495112187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9641022a0 0x7fc964104690 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 -- 172.21.15.111:0/2495112187 >> 172.21.15.111:0/2495112187 conn(0x7fc9640fbc30 msgr2=0x7fc9640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 -- 172.21.15.111:0/2495112187 shutdown_connections 2024-04-24T22:41:47.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.371+0000 7fc96a200640 1 -- 172.21.15.111:0/2495112187 wait complete. 2024-04-24T22:41:47.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.372+0000 7fc96a200640 1 Processor -- start 2024-04-24T22:41:47.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.372+0000 7fc96a200640 1 -- start start 2024-04-24T22:41:47.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc96419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc964107500 0x7fc9641a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d410 con 0x7fc964104bd0 2024-04-24T22:41:47.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d290 con 0x7fc964107500 2024-04-24T22:41:47.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc96a200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d590 con 0x7fc9641022a0 2024-04-24T22:41:47.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc95be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc96419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc95be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc96419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50822/0 (socket says 172.21.15.111:50822) 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:46926/0 (socket says 172.21.15.111:46926) 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc95be00640 1 -- 172.21.15.111:0/3038336690 learned_addr learned my addr 172.21.15.111:0/3038336690 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963e00640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc964107500 0x7fc9641a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc964107500 msgr2=0x7fc9641a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc964107500 0x7fc9641a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 msgr2=0x7fc96419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc96419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.373+0000 7fc963400640 1 -- 172.21.15.111:0/3038336690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9641a1850 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc963400640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc954031440 tx=0x7fc954004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95403f070 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9641a1ae0 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc954003cd0 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954035070 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.374+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9641a1fa0 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.375+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc964108ff0 con 0x7fc9641022a0 2024-04-24T22:41:47.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.377+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fc95403b4f0 con 0x7fc9641022a0 2024-04-24T22:41:47.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.378+0000 7fc960c00640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 0x7fc9480a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.378+0000 7fc95be00640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 msgr2=0x7fc9480a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:47.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.379+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc9540ef860 con 0x7fc9641022a0 2024-04-24T22:41:47.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.379+0000 7fc95be00640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 0x7fc9480a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:47.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.382+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc9540bc120 con 0x7fc9641022a0 2024-04-24T22:41:47.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3876708630' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.408 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.408 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: ignoring --setuser ceph since I am not root 2024-04-24T22:41:47.408 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: ignoring --setgroup ceph since I am not root 2024-04-24T22:41:47.409 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-mgr[33759]: -- 172.21.15.171:0/2833480282 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c7660e5a0 con 0x563c765e2800 2024-04-24T22:41:47.409 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:47.333+0000 7f107fc0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:41:47.476 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setuser ceph since I am not root 2024-04-24T22:41:47.477 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: ignoring --setgroup ceph since I am not root 2024-04-24T22:41:47.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3876708630' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3876708630' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T22:41:47.500 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: ignoring --setuser ceph since I am not root 2024-04-24T22:41:47.500 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: ignoring --setgroup ceph since I am not root 2024-04-24T22:41:47.500 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-mgr[34510]: -- 172.21.15.186:0/1682595464 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fb011c5a0 con 0x564fb00f0800 2024-04-24T22:41:47.500 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:47.338+0000 7f50ab05b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:41:47.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.580+0000 7fc95be00640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 msgr2=0x7fc9480a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:47.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.580+0000 7fc95be00640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 0x7fc9480a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:47.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.582+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc964062340 con 0x7fc9641022a0 2024-04-24T22:41:47.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.590+0000 7fc960c00640 1 -- 172.21.15.111:0/3038336690 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7fc9540c0420 con 0x7fc9641022a0 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 msgr2=0x7fc9480a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 0x7fc9480a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 msgr2=0x7fc964199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc954031440 tx=0x7fc954004270 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 shutdown_connections 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fc9480a2c80 0x7fc9480a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc964107500 0x7fc9641a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc964104bd0 0x7fc96419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 --2- 172.21.15.111:0/3038336690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9641022a0 0x7fc964199c40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 >> 172.21.15.111:0/3038336690 conn(0x7fc9640fbc30 msgr2=0x7fc9641019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:47.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 shutdown_connections 2024-04-24T22:41:47.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.593+0000 7fc96a200640 1 -- 172.21.15.111:0/3038336690 wait complete. 2024-04-24T22:41:47.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-24T22:41:47.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:47.474+0000 7fccb330f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:41:47.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:47.720+0000 7fccb330f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:47.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 -- 172.21.15.111:0/2946657976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 msgr2=0x7f9fe8106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 --2- 172.21.15.111:0/2946657976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe8106fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009860 tx=0x7f9fd002a340 comp rx=0 tx=0).stop 2024-04-24T22:41:47.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 -- 172.21.15.111:0/2946657976 shutdown_connections 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 --2- 172.21.15.111:0/2946657976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 0x7f9fe8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 --2- 172.21.15.111:0/2946657976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe8106fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 --2- 172.21.15.111:0/2946657976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe81046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 -- 172.21.15.111:0/2946657976 >> 172.21.15.111:0/2946657976 conn(0x7f9fe80fbc00 msgr2=0x7f9fe80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 -- 172.21.15.111:0/2946657976 shutdown_connections 2024-04-24T22:41:47.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.791+0000 7f9fee600640 1 -- 172.21.15.111:0/2946657976 wait complete. 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 Processor -- start 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 -- start start 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 0x7f9fe81a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d410 con 0x7f9fe81022b0 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d290 con 0x7f9fe8104be0 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fee600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d590 con 0x7f9fe8107510 2024-04-24T22:41:47.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.793+0000 7f9fdfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36572/0 (socket says 172.21.15.111:36572) 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 -- 172.21.15.111:0/187321821 learned_addr learned my addr 172.21.15.111:0/187321821 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fe7e00640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 0x7f9fe81a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fe7400640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 msgr2=0x7f9fe81a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 0x7f9fe81a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 msgr2=0x7f9fe8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:47.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:47.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 -- 172.21.15.111:0/187321821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81a1870 con 0x7f9fe8104be0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fe7400640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fdfe00640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0035040 tx=0x7f9fd0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.794+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0037040 con 0x7f9fe8104be0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.795+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd0030850 con 0x7f9fe8104be0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.795+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81a1b00 con 0x7f9fe8104be0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.795+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0004920 con 0x7f9fe8104be0 2024-04-24T22:41:47.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.795+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81a2040 con 0x7f9fe8104be0 2024-04-24T22:41:47.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.796+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe8109000 con 0x7f9fe8104be0 2024-04-24T22:41:47.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.797+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f9fd0004050 con 0x7f9fe8104be0 2024-04-24T22:41:47.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.799+0000 7f9fe4c00640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 0x7f9fcc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:47.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.799+0000 7f9fe7400640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 msgr2=0x7f9fcc0a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:47.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.799+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9fd00eeff0 con 0x7f9fe8104be0 2024-04-24T22:41:47.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.799+0000 7f9fe7400640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 0x7f9fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:47.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.803+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9fd00bbf20 con 0x7f9fe8104be0 2024-04-24T22:41:47.906 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:47.517+0000 7f107fc0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:47.907 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:47.618+0000 7f107fc0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:41:47.937 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:47.525+0000 7f50ab05b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:47.937 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:47.627+0000 7f50ab05b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:41:47.937 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:47.935+0000 7f50ab05b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:41:47.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:47.993+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9fe802d170 con 0x7f9fe8104be0 2024-04-24T22:41:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.000+0000 7f9fe4c00640 1 -- 172.21.15.111:0/187321821 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f9fd00bb820 con 0x7f9fe8104be0 2024-04-24T22:41:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.000+0000 7f9fe7400640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 msgr2=0x7f9fcc0a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:48.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.000+0000 7f9fe7400640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 0x7f9fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 msgr2=0x7f9fcc0a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 0x7f9fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 msgr2=0x7f9fe819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0035040 tx=0x7f9fd0033d90 comp rx=0 tx=0).stop 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 shutdown_connections 2024-04-24T22:41:48.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f9fcc0a2d00 0x7f9fcc0a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe8107510 0x7f9fe81a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8104be0 0x7f9fe819a1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 --2- 172.21.15.111:0/187321821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe81022b0 0x7f9fe8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 >> 172.21.15.111:0/187321821 conn(0x7f9fe80fbc00 msgr2=0x7f9fe8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.002+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 shutdown_connections 2024-04-24T22:41:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:48.003+0000 7f9fee600640 1 -- 172.21.15.111:0/187321821 wait complete. 2024-04-24T22:41:48.023 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T22:41:48.047 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T22:41:48.047 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T22:41:48.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:48.048 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-04-24T22:41:48.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:48.100 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-04-24T22:41:48.157 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:47 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:47.863+0000 7fccb330f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:41:48.180 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2024-04-24T22:41:48.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:41:48.187 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-24T22:41:48.187 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:41:48.187 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:48.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T22:41:48.249 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:48.021+0000 7f50ab05b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:41:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:48 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T22:41:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:48 smithi186 ceph-mon[32753]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T22:41:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:48 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:48 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:48 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T22:41:48.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:48 smithi171 ceph-mon[32337]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T22:41:48.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:48 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:48 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.407 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:47.926+0000 7f107fc0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:41:48.407 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:48.011+0000 7f107fc0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:41:48.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:48.268+0000 7fccb330f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:41:48.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:48.382+0000 7fccb330f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:41:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T22:41:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-mon[27746]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T22:41:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:48 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' 2024-04-24T22:41:48.615 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:41:49.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:49.201+0000 7f50ab05b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:41:49.500 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:49.370+0000 7f50ab05b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:49.533 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:49.189+0000 7f107fc0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:41:49.533 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:49.361+0000 7f107fc0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:49.534 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:49.531+0000 7f107fc0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:41:49.834 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:49.542+0000 7f50ab05b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:41:50.119 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:49.821+0000 7f107fc0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:41:50.119 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:49.914+0000 7f107fc0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:41:50.132 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:49.832+0000 7f50ab05b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:41:50.132 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:49.925+0000 7f50ab05b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:41:50.197 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:49.962+0000 7fccb330f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:41:50.406 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:50.117+0000 7f107fc0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:41:50.406 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:50.309+0000 7f107fc0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:50.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:50.130+0000 7f50ab05b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:41:50.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:50.325+0000 7f50ab05b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:50.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:50.198+0000 7fccb330f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:50.534 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:50.417+0000 7fccb330f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 -- 172.21.15.111:0/1618506228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa974103c40 msgr2=0x7fa97410c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 --2- 172.21.15.111:0/1618506228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa974103c40 0x7fa97410c130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa960009990 tx=0x7fa960009ca0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 -- 172.21.15.111:0/1618506228 shutdown_connections 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 --2- 172.21.15.111:0/1618506228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97410c750 0x7fa97410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 --2- 172.21.15.111:0/1618506228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103320 0x7fa974103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 --2- 172.21.15.111:0/1618506228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa974103c40 0x7fa97410c130 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 -- 172.21.15.111:0/1618506228 >> 172.21.15.111:0/1618506228 conn(0x7fa9740fcb50 msgr2=0x7fa9740fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 -- 172.21.15.111:0/1618506228 shutdown_connections 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.726+0000 7fa97a000700 1 -- 172.21.15.111:0/1618506228 wait complete. 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 Processor -- start 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 -- start start 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 0x7fa9741a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 0x7fa9741a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 0x7fa97419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741124c0 con 0x7fa974103c40 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa974112340 con 0x7fa974103320 2024-04-24T22:41:50.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.727+0000 7fa97a000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa974112640 con 0x7fa97410c750 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa973e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 0x7fa9741a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa979600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 0x7fa97419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa979600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 0x7fa97419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:46952/0 (socket says 172.21.15.111:46952) 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa973e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 0x7fa9741a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50858/0 (socket says 172.21.15.111:50858) 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa979600700 1 -- 172.21.15.111:0/2848754499 learned_addr learned my addr 172.21.15.111:0/2848754499 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 0x7fa9741a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 msgr2=0x7fa97419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 0x7fa97419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 msgr2=0x7fa9741a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 0x7fa9741a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:50.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 -- 172.21.15.111:0/2848754499 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97419b140 con 0x7fa974103320 2024-04-24T22:41:50.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.728+0000 7fa978c00700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 0x7fa9741a0820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:50.832 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:50.805+0000 7fccb330f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:41:50.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.729+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96800c9c0 con 0x7fa974103320 2024-04-24T22:41:50.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.729+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96800cb60 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.729+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968018680 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.729+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97419b430 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.729+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a76f0 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.730+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa974068f20 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.732+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fa968010430 con 0x7fa974103320 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.733+0000 7fa971600700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:50.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.733+0000 7fa973e00700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 msgr2=0x7fa95c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:50.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.733+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa9680cf170 con 0x7fa974103320 2024-04-24T22:41:50.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.733+0000 7fa973e00700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:50.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.737+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa96809c080 con 0x7fa974103320 2024-04-24T22:41:50.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.934+0000 7fa973e00700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 msgr2=0x7fa95c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:50.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:50.934+0000 7fa973e00700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:51.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.011+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa97402db30 con 0x7fa974103320 2024-04-24T22:41:51.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.012+0000 7fa971600700 1 -- 172.21.15.111:0/2848754499 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fa96809b980 con 0x7fa974103320 2024-04-24T22:41:51.014 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:41:51.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 msgr2=0x7fa95c0a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:51.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 msgr2=0x7fa9741a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 0x7fa9741a0820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800ed00 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 shutdown_connections 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97410c750 0x7fa97419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa974103320 0x7fa9741a0820 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 --2- 172.21.15.111:0/2848754499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa974103c40 0x7fa9741a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.015+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 >> 172.21.15.111:0/2848754499 conn(0x7fa9740fcb50 msgr2=0x7fa9740fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.016+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 shutdown_connections 2024-04-24T22:41:51.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:51.016+0000 7fa97a000700 1 -- 172.21.15.111:0/2848754499 wait complete. 2024-04-24T22:41:51.201 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:50.932+0000 7fccb330f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:41:51.201 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:51 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2848754499' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:41:51.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:51 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2848754499' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:41:51.250 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:50.928+0000 7f50ab05b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:41:51.338 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:51 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2848754499' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:41:51.338 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:50.908+0000 7f107fc0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:41:51.478 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:51.198+0000 7fccb330f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:41:51.636 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:51.336+0000 7f107fc0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:41:51.636 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:51.433+0000 7f107fc0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:41:51.653 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:51.355+0000 7f50ab05b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:41:51.654 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:51.451+0000 7f50ab05b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:41:51.783 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:51.476+0000 7fccb330f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:41:51.851 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:41:51.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:51.852 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-04-24T22:41:51.898 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:41:51.898 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:41:51.898 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:41:51.898 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T22:41:51.899 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 41944sec preferred_lft 41944sec 2024-04-24T22:41:51.900 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-04-24T22:41:51.900 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:41:51.900 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:41:51.900 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:41:51.900 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-04-24T22:41:51.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:41:51.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-04-24T22:41:51.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-04-24T22:41:51.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:41:51.901 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-04-24T22:41:51.906 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:51.634+0000 7f107fc0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:41:51.999 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:51.651+0000 7f50ab05b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:41:52.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:52.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:52.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:52.390 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:41:52.453 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-04-24T22:41:52.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:52.457 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-04-24T22:41:52.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.179+0000 7f50ab05b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:41:52.499 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.272+0000 7f50ab05b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:41:52.500 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.163+0000 7f107fc0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:41:52.500 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.255+0000 7f107fc0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:41:52.509 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-04-24T22:41:52.509 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:41:52.509 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T22:41:52.509 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T22:41:52.509 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T22:41:52.510 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-04-24T22:41:52.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:52.620 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:52 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:52.320+0000 7fccb330f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:41:52.848 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.498+0000 7f107fc0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:41:52.848 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.633+0000 7f107fc0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:41:52.848 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.729+0000 7f107fc0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:41:52.848 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.846+0000 7f107fc0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:52.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.517+0000 7f50ab05b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:41:52.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.651+0000 7f50ab05b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:41:52.866 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.747+0000 7f50ab05b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:41:52.866 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.863+0000 7f50ab05b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:53.156 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:52.939+0000 7f107fc0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:41:53.249 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:52.957+0000 7f50ab05b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:41:53.262 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:52 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:52.933+0000 7fccb330f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:41:53.262 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:53.065+0000 7fccb330f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:41:53.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:53.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:53.273 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-04-24T22:41:53.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:53.315 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-04-24T22:41:53.403 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:41:53.403 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-04-24T22:41:53.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:41:53.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T22:41:53.404 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-04-24T22:41:53.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:53.514 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:53.337+0000 7fccb330f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:41:53.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:53.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T22:41:53.564 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:41:53.564 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-04-24T22:41:53.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T22:41:53.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:41:53.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T22:41:53.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T22:41:53.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T22:41:53.565 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-04-24T22:41:53.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:53.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:53.862 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:41:53.862 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-04-24T22:41:53.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-04-24T22:41:53.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:41:53.862 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-04-24T22:41:53.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:54.046 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:53.776+0000 7f50ab05b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:41:54.047 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:41:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z[34437]: 2024-04-24T22:41:53.994+0000 7f50ab05b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:41:54.047 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:53.752+0000 7f107fc0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:41:54.048 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:41:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y[33755]: 2024-04-24T22:41:53.967+0000 7f107fc0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:41:54.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:41:54.051 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:54.092 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:41:54.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:41:54.093 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:54.179 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T22:41:54.179 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T22:41:54.179 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/3331d266-e75a-4000-8d95-5eee04103e7a 2024-04-24T22:41:54.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:54.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/3331d266-e75a-4000-8d95-5eee04103e7a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-24T22:41:54.337 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-24T22:41:54.337 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: Standby manager daemon y restarted 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: Standby manager daemon y started 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: Standby manager daemon z restarted 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: Standby manager daemon z started 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:41:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.408 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:41:54.408 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:54 smithi171 ceph-mon[32337]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:54.046+0000 7fccb330f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:54.179+0000 7fccb330f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: Standby manager daemon y restarted 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: Standby manager daemon y started 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.466 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: Standby manager daemon z restarted 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: Standby manager daemon z started 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:41:54.467 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-mon[27746]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: Standby manager daemon y restarted 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: Standby manager daemon y started 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.171:0/520882004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: Standby manager daemon z restarted 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: Standby manager daemon z started 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T22:41:54.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:54 smithi186 ceph-mon[32753]: from='mgr.? 172.21.15.186:0/356459734' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:41:54.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:54.524+0000 7fccb330f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:41:54.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:54.715+0000 7fccb330f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:41:55.122 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:55 smithi111 ceph-mon[27746]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T22:41:55.122 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:54.847+0000 7fccb330f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:41:55.122 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:55 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:55.005+0000 7fccb330f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:41:55.136 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-24T22:41:55.136 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-04-24T22:41:55.136 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.171:6789,172.21.15.186:6789" 2024-04-24T22:41:55.136 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with new device syntax: 0@0d3ba9b8-028a-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/3331d266-e75a-4000-8d95-5eee04103e7a 2024-04-24T22:41:55.136 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.111:6789/172.21.15.171:6789/172.21.15.186:6789" will pass to kernel 2024-04-24T22:41:55.142 INFO:teuthology.orchestra.run.smithi111.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T22:41:55.142 INFO:teuthology.orchestra.run.smithi111.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:41:55.142 INFO:teuthology.orchestra.run.smithi111.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:41:55.142 INFO:teuthology.orchestra.run.smithi111.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:41:55.142 INFO:teuthology.orchestra.run.smithi111.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:41:55.144 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:41:55.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:41:55.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:55.212 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:41:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:41:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:55.325 INFO:teuthology.orchestra.run.smithi111.stdout:client25225 2024-04-24T22:41:55.326 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:41:55.326 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:41:55.373 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:41:55.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:55 smithi171 ceph-mon[32337]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T22:41:55.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:55 smithi186 ceph-mon[32753]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T22:41:55.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:41:55.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:55.528 INFO:teuthology.orchestra.run.smithi111.stdout:client25225 2024-04-24T22:41:55.529 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:41:55.533 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:55 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:55.136+0000 7fccb330f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:41:55.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:41:55.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:55.635 INFO:teuthology.orchestra.run.smithi111.stdout:client25225 2024-04-24T22:41:55.636 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968.client25225/status') 2024-04-24T22:41:55.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:41:55.759 INFO:teuthology.orchestra.run.smithi111.stdout:instance: client.25225 (3)172.21.15.111:0/4150889195 2024-04-24T22:41:55.760 INFO:teuthology.orchestra.run.smithi111.stdout:blocklisted: false 2024-04-24T22:41:55.760 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records in 2024-04-24T22:41:55.760 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records out 2024-04-24T22:41:55.760 INFO:teuthology.orchestra.run.smithi111.stderr:72 bytes copied, 9.3386e-05 s, 771 kB/s 2024-04-24T22:41:55.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:41:55.770 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T22:41:55.780 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T22:41:55.780 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:55.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 -- 172.21.15.111:0/1135812892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 msgr2=0x7f648c1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 --2- 172.21.15.111:0/1135812892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1019f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6480009900 tx=0x7f648002a410 comp rx=0 tx=0).stop 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 -- 172.21.15.111:0/1135812892 shutdown_connections 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 --2- 172.21.15.111:0/1135812892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 0x7f648c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 --2- 172.21.15.111:0/1135812892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 0x7f648c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 --2- 172.21.15.111:0/1135812892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1019f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 -- 172.21.15.111:0/1135812892 >> 172.21.15.111:0/1135812892 conn(0x7f648c0fcb40 msgr2=0x7f648c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 -- 172.21.15.111:0/1135812892 shutdown_connections 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.960+0000 7f6492600640 1 -- 172.21.15.111:0/1135812892 wait complete. 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 Processor -- start 2024-04-24T22:41:55.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 -- start start 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 0x7f648c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 0x7f648c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f648c10d2a0 con 0x7f648c1015d0 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f648c10d120 con 0x7f648c1027c0 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.961+0000 7f6492600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f648c10d420 con 0x7f648c103180 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58096/0 (socket says 172.21.15.111:58096) 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 -- 172.21.15.111:0/982102593 learned_addr learned my addr 172.21.15.111:0/982102593 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491c00640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 0x7f648c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f648be00640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 0x7f648c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 msgr2=0x7f648c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 0x7f648c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 msgr2=0x7f648c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 0x7f648c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.962+0000 7f6491200640 1 -- 172.21.15.111:0/982102593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19f510 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6491200640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6480031440 tx=0x7f6480032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648003f030 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6480003ce0 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c19f7a0 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480035070 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.963+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c19fce0 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.964+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c108970 con 0x7f648c1015d0 2024-04-24T22:41:55.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.967+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137851+0+0 (secure 0 0 0) 0x7f6480042030 con 0x7f648c1015d0 2024-04-24T22:41:55.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.969+0000 7f6489600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 0x7f64740a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.969+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f64800ef500 con 0x7f648c1015d0 2024-04-24T22:41:55.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.969+0000 7f648be00640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 msgr2=0x7f64740a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:55.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.969+0000 7f648be00640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 0x7f64740a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:55.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:55.974+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f64800bbdc0 con 0x7f648c1015d0 2024-04-24T22:41:56.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.170+0000 7f648be00640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 msgr2=0x7f64740a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/1872264645 2024-04-24T22:41:56.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.170+0000 7f648be00640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 0x7f64740a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:56.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.216+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f648c02d170 con 0x7f648c1015d0 2024-04-24T22:41:56.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.270+0000 7f6489600640 1 -- 172.21.15.111:0/982102593 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f64800c00c0 con 0x7f648c1015d0 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 msgr2=0x7f64740a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 0x7f64740a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 msgr2=0x7f648c1978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6480031440 tx=0x7f6480032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 shutdown_connections 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:6800/1872264645,v1:172.21.15.111:6801/1872264645] conn(0x7f64740a2c50 0x7f64740a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f648c103180 0x7f648c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f648c1027c0 0x7f648c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 --2- 172.21.15.111:0/982102593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f648c1015d0 0x7f648c1978f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 >> 172.21.15.111:0/982102593 conn(0x7f648c0fcb40 msgr2=0x7f648c105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 shutdown_connections 2024-04-24T22:41:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.274+0000 7f6492600640 1 -- 172.21.15.111:0/982102593 wait complete. 2024-04-24T22:41:56.293 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-24T22:41:56.481 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.187+0000 7fccb330f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:41:56.482 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1448666231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 msgr2=0x7fb3dc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1448666231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1015b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009ca0 tx=0x7fb3d402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1448666231 shutdown_connections 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1448666231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1015b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1448666231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 0x7fb3dc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1448666231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1448666231 >> 172.21.15.111:0/1448666231 conn(0x7fb3dc0fba30 msgr2=0x7fb3dc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:56.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1448666231 shutdown_connections 2024-04-24T22:41:56.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.486+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1448666231 wait complete. 2024-04-24T22:41:56.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.487+0000 7fb3dbe00640 1 Processor -- start 2024-04-24T22:41:56.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 -- start start 2024-04-24T22:41:56.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 0x7fb3dc199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10f460 con 0x7fb3dc101130 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10f2e0 con 0x7fb3dc100790 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3dbe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10f5e0 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3daa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3daa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34026/0 (socket says 172.21.15.111:34026) 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3daa00640 1 -- 172.21.15.111:0/1490756308 learned_addr learned my addr 172.21.15.111:0/1490756308 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3d3e00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 0x7fb3dc199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.488+0000 7fb3db400640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 -- 172.21.15.111:0/1490756308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 msgr2=0x7fb3dc199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:56.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 0x7fb3dc199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 -- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 msgr2=0x7fb3dc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:56.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:56.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 -- 172.21.15.111:0/1490756308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc1a1640 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.489+0000 7fb3daa00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00d950 tx=0x7fb3cc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:56.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.501+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc017030 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.501+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc004020 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.501+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003d60 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.501+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc1a1930 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.502+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc1a1e30 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.503+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc10ac90 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.506+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7fb3cc01a070 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.507+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fb3cc0ce730 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:56.511+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3cc09b940 con 0x7fb3dc0ffe30 2024-04-24T22:41:56.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:56 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T22:41:56.694 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:56 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T22:41:56.784 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.479+0000 7fccb330f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:41:57.282 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.837+0000 7fcc08400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.282 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.837+0000 7fcc08400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.282 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.837+0000 7fcc08400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.837+0000 7fcc08400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.837+0000 7fcc08400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.838+0000 7fcc05200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.838+0000 7fcc05200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.838+0000 7fcc05200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.838+0000 7fcc05200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.283 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:56.838+0000 7fcc05200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:57.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.512+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fb3cc096f90 con 0x7fb3dc0ffe30 2024-04-24T22:41:57.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.515+0000 7fb3d3400640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 0x7fb3c00abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.515+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb3cc018e50 con 0x7fb3c00a9900 2024-04-24T22:41:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.517+0000 7fb3d3e00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 0x7fb3c00abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.524+0000 7fb3d3e00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 0x7fb3c00abcf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc19af30 tx=0x7fb3c400b390 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.538 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: Active manager daemon x restarted 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: Activating manager daemon x 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: mgrmap e21: x(active, starting, since 0.0122389s), standbys: y, z 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:41:57.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:41:57.540 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: Manager daemon x is now available 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.541 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.542 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.549+0000 7fb3d3400640 1 -- 172.21.15.111:0/1490756308 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb3cc018e50 con 0x7fb3c00a9900 2024-04-24T22:41:57.551 INFO:teuthology.orchestra.run.smithi111.stdout:{} 2024-04-24T22:41:57.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.551+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 msgr2=0x7fb3c00abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.551+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 0x7fb3c00abcf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc19af30 tx=0x7fb3c400b390 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 msgr2=0x7fb3dc199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00d950 tx=0x7fb3cc00de10 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 shutdown_connections 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3dc101130 0x7fb3dc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc100790 0x7fb3dc199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb3c00a9900 0x7fb3c00abcf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 --2- 172.21.15.111:0/1490756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3dc0ffe30 0x7fb3dc199a20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 >> 172.21.15.111:0/1490756308 conn(0x7fb3dc0fba30 msgr2=0x7fb3dc0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 shutdown_connections 2024-04-24T22:41:57.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.552+0000 7fb3dbe00640 1 -- 172.21.15.111:0/1490756308 wait complete. 2024-04-24T22:41:57.573 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T22:41:57.585 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T22:41:57.602 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T22:41:57.602 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:57.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: Active manager daemon x restarted 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: Activating manager daemon x 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: mgrmap e21: x(active, starting, since 0.0122389s), standbys: y, z 2024-04-24T22:41:57.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:41:57.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:41:57.659 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: Manager daemon x is now available 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.660 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:57 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/982102593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: Active manager daemon x restarted 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: Activating manager daemon x 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: mgrmap e21: x(active, starting, since 0.0122389s), standbys: y, z 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T22:41:57.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:41:57.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: Manager daemon x is now available 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:41:57.754 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.754 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:57 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T22:41:57.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 -- 172.21.15.111:0/2512505032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81049f0 msgr2=0x7fa0f8104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 --2- 172.21.15.111:0/2512505032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81049f0 0x7fa0f8104e70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009900 tx=0x7fa0e402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 -- 172.21.15.111:0/2512505032 shutdown_connections 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 --2- 172.21.15.111:0/2512505032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f81053b0 0x7fa0f810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 --2- 172.21.15.111:0/2512505032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81049f0 0x7fa0f8104e70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 --2- 172.21.15.111:0/2512505032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f8103800 0x7fa0f8103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 -- 172.21.15.111:0/2512505032 >> 172.21.15.111:0/2512505032 conn(0x7fa0f80fed70 msgr2=0x7fa0f81011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 -- 172.21.15.111:0/2512505032 shutdown_connections 2024-04-24T22:41:57.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.781+0000 7fa0fd000640 1 -- 172.21.15.111:0/2512505032 wait complete. 2024-04-24T22:41:57.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.782+0000 7fa0fd000640 1 Processor -- start 2024-04-24T22:41:57.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.782+0000 7fa0fd000640 1 -- start start 2024-04-24T22:41:57.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 0x7fa0f8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81053b0 0x7fa0f81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810f4d0 con 0x7fa0f8103800 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810f350 con 0x7fa0f81049f0 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0fd000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810f650 con 0x7fa0f81053b0 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 0x7fa0f8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:39260/0 (socket says 172.21.15.111:39260) 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81053b0 0x7fa0f81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 -- 172.21.15.111:0/4100284436 learned_addr learned my addr 172.21.15.111:0/4100284436 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 -- 172.21.15.111:0/4100284436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81053b0 msgr2=0x7fa0f81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81053b0 0x7fa0f81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 -- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 msgr2=0x7fa0f8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 0x7fa0f8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f6a00640 1 -- 172.21.15.111:0/4100284436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81a16d0 con 0x7fa0f81049f0 2024-04-24T22:41:57.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.783+0000 7fa0f7400640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 0x7fa0f8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:57.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0f6a00640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4031440 tx=0x7fa0e4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e403f030 con 0x7fa0f81049f0 2024-04-24T22:41:57.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4003ce0 con 0x7fa0f81049f0 2024-04-24T22:41:57.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4035070 con 0x7fa0f81049f0 2024-04-24T22:41:57.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f81a1960 con 0x7fa0f81049f0 2024-04-24T22:41:57.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.784+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f81a1ea0 con 0x7fa0f81049f0 2024-04-24T22:41:57.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.785+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f810ab20 con 0x7fa0f81049f0 2024-04-24T22:41:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.788+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fa0e4042030 con 0x7fa0f81049f0 2024-04-24T22:41:57.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.789+0000 7fa0efe00640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 0x7fa0dc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.789+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fa0e40ef1c0 con 0x7fa0f81049f0 2024-04-24T22:41:57.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.789+0000 7fa0f7400640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 0x7fa0dc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.790+0000 7fa0f7400640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 0x7fa0dc0a4eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8104850 tx=0x7fa0e8005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:57.793+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa0e40bbcd0 con 0x7fa0f81049f0 2024-04-24T22:41:58.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.026+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa0f8061b80 con 0x7fa0dc0a29f0 2024-04-24T22:41:58.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:57.535+0000 7fcc02000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:58.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:57.535+0000 7fcc02000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:58.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:57.535+0000 7fcc02000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:58.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:57.535+0000 7fcc02000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:58.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:41:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:41:57.535+0000 7fcc02000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:41:58.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.138+0000 7fa0efe00640 1 -- 172.21.15.111:0/4100284436 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa0f8061b80 con 0x7fa0dc0a29f0 2024-04-24T22:41:58.140 INFO:teuthology.orchestra.run.smithi111.stdout:Schedule set for path / 2024-04-24T22:41:58.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.141+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 msgr2=0x7fa0dc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.141+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 0x7fa0dc0a4eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8104850 tx=0x7fa0e8005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.141+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 msgr2=0x7fa0f8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.141+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4031440 tx=0x7fa0e4032ce0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 shutdown_connections 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa0dc0a29f0 0x7fa0dc0a4eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f81053b0 0x7fa0f81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f81049f0 0x7fa0f8199ff0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 --2- 172.21.15.111:0/4100284436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f8103800 0x7fa0f8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 >> 172.21.15.111:0/4100284436 conn(0x7fa0f80fed70 msgr2=0x7fa0f8107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 shutdown_connections 2024-04-24T22:41:58.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.142+0000 7fa0fd000640 1 -- 172.21.15.111:0/4100284436 wait complete. 2024-04-24T22:41:58.161 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-24T22:41:58.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 -- 172.21.15.111:0/1601227430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 msgr2=0x7fc818102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1601227430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc818102260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009ca0 tx=0x7fc80c02a8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 -- 172.21.15.111:0/1601227430 shutdown_connections 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1601227430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc818105150 0x7fc8181055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1601227430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8181027a0 0x7fc818104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1601227430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc818102260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 -- 172.21.15.111:0/1601227430 >> 172.21.15.111:0/1601227430 conn(0x7fc8180fba30 msgr2=0x7fc8180fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.334+0000 7fc81ee00640 1 -- 172.21.15.111:0/1601227430 shutdown_connections 2024-04-24T22:41:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.335+0000 7fc81ee00640 1 -- 172.21.15.111:0/1601227430 wait complete. 2024-04-24T22:41:58.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 Processor -- start 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 -- start start 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8181027a0 0x7fc81819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc818105150 0x7fc8181a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc818113880 con 0x7fc8181027a0 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc818113700 con 0x7fc818105150 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81ee00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc818113a00 con 0x7fc8180ffe30 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34192/0 (socket says 172.21.15.111:34192) 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81da00640 1 -- 172.21.15.111:0/1281235547 learned_addr learned my addr 172.21.15.111:0/1281235547 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.336+0000 7fc81d000640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8181027a0 0x7fc81819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81e400640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc818105150 0x7fc8181a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 -- 172.21.15.111:0/1281235547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc818105150 msgr2=0x7fc8181a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc818105150 0x7fc8181a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 -- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8181027a0 msgr2=0x7fc81819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8181027a0 0x7fc81819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 -- 172.21.15.111:0/1281235547 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8181a5b40 con 0x7fc8180ffe30 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc81da00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000f80 tx=0x7fc80c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003c70 con 0x7fc8180ffe30 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.337+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc80c003e10 con 0x7fc8180ffe30 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.338+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8181a5dd0 con 0x7fc8180ffe30 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.338+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c02fd50 con 0x7fc8180ffe30 2024-04-24T22:41:58.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.338+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8181a6240 con 0x7fc8180ffe30 2024-04-24T22:41:58.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.339+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8180fd520 con 0x7fc8180ffe30 2024-04-24T22:41:58.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.341+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fc80c004040 con 0x7fc8180ffe30 2024-04-24T22:41:58.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.342+0000 7fc816000640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 0x7fc8000a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.343+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fc80c0f02e0 con 0x7fc8180ffe30 2024-04-24T22:41:58.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.343+0000 7fc81d000640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 0x7fc8000a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.343+0000 7fc81d000640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 0x7fc8000a4eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc81819f430 tx=0x7fc80400b390 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.347+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc80c0bd490 con 0x7fc8180ffe30 2024-04-24T22:41:58.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.595+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc818061c20 con 0x7fc8000a29f0 2024-04-24T22:41:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: mgrmap e22: x(active, since 1.03766s), standbys: y, z 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: [24/Apr/2024:22:41:57] ENGINE Bus STARTING 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: [24/Apr/2024:22:41:57] ENGINE Serving on http://172.21.15.111:8765 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: [24/Apr/2024:22:41:57] ENGINE Serving on https://172.21.15.111:7150 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: [24/Apr/2024:22:41:57] ENGINE Bus STARTED 2024-04-24T22:41:58.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:58 smithi171 ceph-mon[32337]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:58.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.680+0000 7fc816000640 1 -- 172.21.15.111:0/1281235547 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc818061c20 con 0x7fc8000a29f0 2024-04-24T22:41:58.682 INFO:teuthology.orchestra.run.smithi111.stdout:Retention added to path / 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 msgr2=0x7fc8000a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 0x7fc8000a4eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc81819f430 tx=0x7fc80400b390 comp rx=0 tx=0).stop 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 msgr2=0x7fc81819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000f80 tx=0x7fc80c0319f0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 shutdown_connections 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc818105150 0x7fc8181a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8181027a0 0x7fc81819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fc8000a29f0 0x7fc8000a4eb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 --2- 172.21.15.111:0/1281235547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8180ffe30 0x7fc81819df70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 >> 172.21.15.111:0/1281235547 conn(0x7fc8180fba30 msgr2=0x7fc818103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 shutdown_connections 2024-04-24T22:41:58.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.683+0000 7fc81ee00640 1 -- 172.21.15.111:0/1281235547 wait complete. 2024-04-24T22:41:58.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T22:41:58.754 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: mgrmap e22: x(active, since 1.03766s), standbys: y, z 2024-04-24T22:41:58.754 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: [24/Apr/2024:22:41:57] ENGINE Bus STARTING 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: [24/Apr/2024:22:41:57] ENGINE Serving on http://172.21.15.111:8765 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: [24/Apr/2024:22:41:57] ENGINE Serving on https://172.21.15.111:7150 2024-04-24T22:41:58.755 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: [24/Apr/2024:22:41:57] ENGINE Bus STARTED 2024-04-24T22:41:58.756 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:58 smithi111 ceph-mon[27746]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:58.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 -- 172.21.15.111:0/2320534819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c107dd0 msgr2=0x7fd81c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 --2- 172.21.15.111:0/2320534819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c107dd0 0x7fd81c1081f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009900 tx=0x7fd80c02a410 comp rx=0 tx=0).stop 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 -- 172.21.15.111:0/2320534819 shutdown_connections 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 --2- 172.21.15.111:0/2320534819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c109980 0x7fd81c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 --2- 172.21.15.111:0/2320534819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 0x7fd81c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 --2- 172.21.15.111:0/2320534819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c107dd0 0x7fd81c1081f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 -- 172.21.15.111:0/2320534819 >> 172.21.15.111:0/2320534819 conn(0x7fd81c076d90 msgr2=0x7fd81c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 -- 172.21.15.111:0/2320534819 shutdown_connections 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.871+0000 7fd820a00640 1 -- 172.21.15.111:0/2320534819 wait complete. 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.872+0000 7fd820a00640 1 Processor -- start 2024-04-24T22:41:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.872+0000 7fd820a00640 1 -- start start 2024-04-24T22:41:58.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.872+0000 7fd820a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.872+0000 7fd820a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 0x7fd81c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.872+0000 7fd820a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c109980 0x7fd81c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd820a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113a20 con 0x7fd81c108fc0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd820a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1138a0 con 0x7fd81c109980 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd820a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113ba0 con 0x7fd81c107dd0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:34224/0 (socket says 172.21.15.111:34224) 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 -- 172.21.15.111:0/2809922546 learned_addr learned my addr 172.21.15.111:0/2809922546 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819400640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 0x7fd81c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd81a800640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c109980 0x7fd81c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 -- 172.21.15.111:0/2809922546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c109980 msgr2=0x7fd81c1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c109980 0x7fd81c1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 -- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 msgr2=0x7fd81c19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 0x7fd81c19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 -- 172.21.15.111:0/2809922546 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a5c50 con 0x7fd81c107dd0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.873+0000 7fd819e00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd80c031610 tx=0x7fd80c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.874+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c03f030 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.874+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80c003ad0 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.874+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a5ee0 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.874+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c035070 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.874+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a62f0 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.875+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c10f090 con 0x7fd81c107dd0 2024-04-24T22:41:58.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.877+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) ==== 138005+0+0 (secure 0 0 0) 0x7fd80c042030 con 0x7fd81c107dd0 2024-04-24T22:41:58.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.878+0000 7fd812a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 0x7fd8000a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.878+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fd80c0efbd0 con 0x7fd81c107dd0 2024-04-24T22:41:58.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.878+0000 7fd819400640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 0x7fd8000a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.879+0000 7fd819400640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 0x7fd8000a5300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd81c19f540 tx=0x7fd808005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:58.883+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd80c0bc000 con 0x7fd81c107dd0 2024-04-24T22:41:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: mgrmap e22: x(active, since 1.03766s), standbys: y, z 2024-04-24T22:41:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: [24/Apr/2024:22:41:57] ENGINE Bus STARTING 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: [24/Apr/2024:22:41:57] ENGINE Serving on http://172.21.15.111:8765 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: [24/Apr/2024:22:41:57] ENGINE Serving on https://172.21.15.111:7150 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: [24/Apr/2024:22:41:57] ENGINE Bus STARTED 2024-04-24T22:41:59.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:58 smithi186 ceph-mon[32753]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.100+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd81c061b80 con 0x7fd8000a2e40 2024-04-24T22:41:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.110+0000 7fd812a00640 1 -- 172.21.15.111:0/2809922546 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fd81c061b80 con 0x7fd8000a2e40 2024-04-24T22:41:59.113 INFO:teuthology.orchestra.run.smithi111.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T22:41:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-24T22:41:59.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 msgr2=0x7fd8000a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 0x7fd8000a5300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd81c19f540 tx=0x7fd808005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:59.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 msgr2=0x7fd81c19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd80c031610 tx=0x7fd80c004270 comp rx=0 tx=0).stop 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 shutdown_connections 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd81c109980 0x7fd81c1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c108fc0 0x7fd81c19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd8000a2e40 0x7fd8000a5300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 --2- 172.21.15.111:0/2809922546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c107dd0 0x7fd81c19e030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 >> 172.21.15.111:0/2809922546 conn(0x7fd81c076d90 msgr2=0x7fd81c10c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 shutdown_connections 2024-04-24T22:41:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.113+0000 7fd820a00640 1 -- 172.21.15.111:0/2809922546 wait complete. 2024-04-24T22:41:59.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T22:41:59.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 -- 172.21.15.111:0/1680095250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 msgr2=0x7f5084110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 --2- 172.21.15.111:0/1680095250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f5084110aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f507c009cd0 tx=0x7f507c02a8e0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 -- 172.21.15.111:0/1680095250 shutdown_connections 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 --2- 172.21.15.111:0/1680095250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f5084110aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 --2- 172.21.15.111:0/1680095250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f5084109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 --2- 172.21.15.111:0/1680095250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f5084108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 -- 172.21.15.111:0/1680095250 >> 172.21.15.111:0/1680095250 conn(0x7f508406e970 msgr2=0x7f5084070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 -- 172.21.15.111:0/1680095250 shutdown_connections 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.320+0000 7f5089e00640 1 -- 172.21.15.111:0/1680095250 wait complete. 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.321+0000 7f5089e00640 1 Processor -- start 2024-04-24T22:41:59.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 -- start start 2024-04-24T22:41:59.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f50841a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f50841a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5084114b30 con 0x7f5084109850 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f50841149b0 con 0x7f5084108a50 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5089e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5084114cb0 con 0x7f508410a210 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5082a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f507be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f50841a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5082a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:39334/0 (socket says 172.21.15.111:39334) 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f507be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f50841a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58288/0 (socket says 172.21.15.111:58288) 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5082a00640 1 -- 172.21.15.111:0/3017982560 learned_addr learned my addr 172.21.15.111:0/3017982560 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.322+0000 7f5083400640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f50841a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 -- 172.21.15.111:0/3017982560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 msgr2=0x7f50841a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f50841a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 -- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 msgr2=0x7f50841a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f50841a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 -- 172.21.15.111:0/3017982560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a7df0 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5082a00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f507400e9f0 tx=0x7f507400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507400cd80 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5074010910 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a80e0 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074018990 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.323+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a8680 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.324+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50841a88e0 con 0x7f5084108a50 2024-04-24T22:41:59.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.326+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 23) ==== 138005+0+0 (secure 0 0 0) 0x7f5074010ac0 con 0x7f5084108a50 2024-04-24T22:41:59.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.327+0000 7f5088a00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 0x7f50640a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.328+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f50740cfd90 con 0x7f5084108a50 2024-04-24T22:41:59.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.328+0000 7f507be00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 0x7f50640a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.328+0000 7f507be00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 0x7f50640a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50841a16e0 tx=0x7f506c005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:59.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.330+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f507409c1c0 con 0x7f5084108a50 2024-04-24T22:41:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.542+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5084070a70 con 0x7f50640a2ec0 2024-04-24T22:41:59.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.551+0000 7f5088a00640 1 -- 172.21.15.111:0/3017982560 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5084070a70 con 0x7f50640a2ec0 2024-04-24T22:41:59.554 INFO:teuthology.orchestra.run.smithi111.stdout:/ 1m 6m3h 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 msgr2=0x7f50640a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 0x7f50640a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50841a16e0 tx=0x7f506c005f90 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 msgr2=0x7f50841a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f507400e9f0 tx=0x7f507400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 shutdown_connections 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f50640a2ec0 0x7f50640a5380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f508410a210 0x7f50841a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5084109850 0x7f50841a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 --2- 172.21.15.111:0/3017982560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5084108a50 0x7f50841a01d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 >> 172.21.15.111:0/3017982560 conn(0x7f508406e970 msgr2=0x7f5084070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 shutdown_connections 2024-04-24T22:41:59.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:41:59.554+0000 7f5089e00640 1 -- 172.21.15.111:0/3017982560 wait complete. 2024-04-24T22:41:59.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-24T22:41:59.642 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-24T22:41:59.674 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-24T22:41:59.704 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:41:59.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-24T22:41:59.720 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T22:41:59.748 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:41:59.748 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T22:41:59.748 INFO:tasks.workunit:timeout=3h 2024-04-24T22:41:59.748 INFO:tasks.workunit:cleanup=True 2024-04-24T22:41:59.748 DEBUG:teuthology.orchestra.run.smithi111:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T22:41:59.800 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-24 22:39:37.943527179 +0000 2024-04-24T22:41:59.801 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-24 22:39:37.943527179 +0000 2024-04-24T22:41:59.801 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-24 22:41:55.205260033 +0000 2024-04-24T22:41:59.801 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-24 22:39:37.943527179 +0000 2024-04-24T22:41:59.801 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T22:41:59.801 DEBUG:teuthology.orchestra.run.smithi111:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T22:41:59.904 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:41:59 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:41:59.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.910 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:41:59 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:41:59.924 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:41:59.968 INFO:tasks.workunit.client.0.smithi111.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:00.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:41:59 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:00 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:42:00.696+0000 7fcbd4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:42:01.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:42:00.696+0000 7fcbd4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:42:01.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:42:00.696+0000 7fcbd4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:42:01.034 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:42:00.696+0000 7fcbd4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:42:00.696+0000 7fcbd4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:00 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.071 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.072 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.073 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.073 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.073 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:01.073 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:01.073 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:00 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:02 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:42:02.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:02 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:42:02.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:02 smithi171 ceph-mon[32337]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:42:02.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:02 smithi171 ceph-mon[32337]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-24T22:42:02.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:02 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:42:02.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:02 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:42:02.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:02 smithi186 ceph-mon[32753]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:42:02.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:02 smithi186 ceph-mon[32753]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-24T22:42:02.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:02 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:42:02.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:02 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:42:02.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:02 smithi111 ceph-mon[27746]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T22:42:02.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:02 smithi111 ceph-mon[27746]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-24T22:42:03.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-24T22:42:03.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:03.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:03.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-24T22:42:03.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:03.909 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:03 smithi171 ceph-mon[32337]: mgrmap e25: x(active, since 6s), standbys: y, z 2024-04-24T22:42:03.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-24T22:42:04.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:04.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:04.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:42:04.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:03 smithi186 ceph-mon[32753]: mgrmap e25: x(active, since 6s), standbys: y, z 2024-04-24T22:42:04.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:42:04.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:03 smithi111 ceph-mon[27746]: mgrmap e25: x(active, since 6s), standbys: y, z 2024-04-24T22:42:05.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:04 smithi111 ceph-mon[27746]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:04 smithi171 ceph-mon[32337]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: Updating smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: Updating smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:05.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:04 smithi186 ceph-mon[32753]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:42:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-04-24T22:42:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:06.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:06.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:05 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:06.113 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:05 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: Updating smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:05 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:07.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:06 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:07.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:06 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:07.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:06 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:42:08.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:07 smithi171 ceph-mon[32337]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-24T22:42:08.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:07 smithi186 ceph-mon[32753]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-24T22:42:08.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:07 smithi111 ceph-mon[27746]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-24T22:42:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:09 smithi171 ceph-mon[32337]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-24T22:42:10.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:09 smithi186 ceph-mon[32753]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-24T22:42:10.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:09 smithi111 ceph-mon[27746]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-24T22:42:12.066 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:11 smithi171 ceph-mon[32337]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-24T22:42:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:11 smithi186 ceph-mon[32753]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-24T22:42:12.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:11 smithi111 ceph-mon[27746]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-24T22:42:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 -- 172.21.15.111:0/1653189797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 msgr2=0x7f283c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 --2- 172.21.15.111:0/1653189797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c103c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2824009890 tx=0x7f282402a360 comp rx=0 tx=0).stop 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 -- 172.21.15.111:0/1653189797 shutdown_connections 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 --2- 172.21.15.111:0/1653189797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 0x7f283c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 --2- 172.21.15.111:0/1653189797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c103c00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 --2- 172.21.15.111:0/1653189797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 0x7f283c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.949+0000 7f2841600640 1 -- 172.21.15.111:0/1653189797 >> 172.21.15.111:0/1653189797 conn(0x7f283c0ff250 msgr2=0x7f283c101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.950+0000 7f2841600640 1 -- 172.21.15.111:0/1653189797 shutdown_connections 2024-04-24T22:42:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.950+0000 7f2841600640 1 -- 172.21.15.111:0/1653189797 wait complete. 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 Processor -- start 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 -- start start 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 0x7f283c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 0x7f283c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f283c113ba0 con 0x7f283c109790 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f283c113a20 con 0x7f283c104140 2024-04-24T22:42:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.951+0000 7f2841600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f283c113d20 con 0x7f283c103780 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50882/0 (socket says 172.21.15.111:50882) 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 -- 172.21.15.111:0/3651830168 learned_addr learned my addr 172.21.15.111:0/3651830168 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f2840c00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 0x7f283c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f2833e00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 0x7f283c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 -- 172.21.15.111:0/3651830168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 msgr2=0x7f283c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 0x7f283c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 -- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 msgr2=0x7f283c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 0x7f283c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 -- 172.21.15.111:0/3651830168 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c19f050 con 0x7f283c103780 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.952+0000 7f283be00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e9f0 tx=0x7f282c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.953+0000 7f2839600640 1 -- 172.21.15.111:0/3651830168 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c00cd80 con 0x7f283c103780 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.953+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19f340 con 0x7f283c103780 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.953+0000 7f2839600640 1 -- 172.21.15.111:0/3651830168 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282c010910 con 0x7f283c103780 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.953+0000 7f2839600640 1 -- 172.21.15.111:0/3651830168 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c0188f0 con 0x7f283c103780 2024-04-24T22:42:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.953+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a6bd0 con 0x7f283c103780 2024-04-24T22:42:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.956+0000 7f2845a66740 1 Processor -- start 2024-04-24T22:42:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.956+0000 7f2839600640 1 -- 172.21.15.111:0/3651830168 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f282c010ac0 con 0x7f283c103780 2024-04-24T22:42:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.956+0000 7f2845a66740 1 -- start start 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752174c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752177580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617520cdab0 con 0x561752175190 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561752076640 con 0x561752169fb0 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2845a66740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5617520c9a70 con 0x56175216a8d0 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2833e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752177580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:49862/0 (socket says 172.21.15.111:49862) 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 -- 172.21.15.111:0/468026816 learned_addr learned my addr 172.21.15.111:0/468026816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f283be00640 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752174c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 -- 172.21.15.111:0/468026816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 msgr2=0x561752174c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752174c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 -- 172.21.15.111:0/468026816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 msgr2=0x561752177580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752177580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.957+0000 7f2840c00640 1 -- 172.21.15.111:0/468026816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561751de2210 con 0x561752169fb0 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2839600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 0x7f28200a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2840c00640 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f283400e580 tx=0x7f28340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d56fcbc7e97615 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2839600640 1 -- 172.21.15.111:0/3651830168 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f282c0cfa40 con 0x7f283c103780 2024-04-24T22:42:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f283be00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 0x7f28200a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2831200640 1 -- 172.21.15.111:0/468026816 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834003970 con 0x561752169fb0 2024-04-24T22:42:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2831200640 1 -- 172.21.15.111:0/468026816 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283400d830 con 0x561752169fb0 2024-04-24T22:42:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.958+0000 7f2831200640 1 -- 172.21.15.111:0/468026816 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834004500 con 0x561752169fb0 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 -- 172.21.15.111:0/468026816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 msgr2=0x56175216a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f283be00640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 0x7f28200a5360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f282c0ccfe0 tx=0x7f282c0d34f0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f283400e580 tx=0x7f28340095c0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 -- 172.21.15.111:0/468026816 shutdown_connections 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752177580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752174c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 --2- 172.21.15.111:0/468026816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752169fb0 0x56175216a390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 -- 172.21.15.111:0/468026816 >> 172.21.15.111:0/468026816 conn(0x5617521664c0 msgr2=0x5617521668a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 -- 172.21.15.111:0/468026816 shutdown_connections 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.959+0000 7f2845a66740 1 -- 172.21.15.111:0/468026816 wait complete. 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 -- start start 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 Processor -- start 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752172620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752175190 0x561752172b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561752177dc0 con 0x561752169fb0 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561752089990 con 0x561752175190 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561752177f40 con 0x56175216a8d0 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.960+0000 7f2845a66740 10 client.0 _ll_register_callbacks cb 0x56175208b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2833e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752172620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43296/0 (socket says 172.21.15.111:43296) 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 -- 172.21.15.111:0/1285333852 learned_addr learned my addr 172.21.15.111:0/1285333852 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752175190 0x561752172b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 -- 172.21.15.111:0/1285333852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 msgr2=0x561752172620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752172620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 -- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752175190 msgr2=0x561752172b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752175190 0x561752172b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 -- 172.21.15.111:0/1285333852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561751eae620 con 0x561752169fb0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2840c00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283400dee0 tx=0x7f2834005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d69f18a373c1e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.961+0000 7f2829600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834004410 con 0x561752169fb0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f28340045b0 con 0x561752169fb0 2024-04-24T22:42:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283400d830 con 0x561752169fb0 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2834035050 con 0x561752169fb0 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2845a66740 10 client.15219 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56175218c450 con 0x561752169fb0 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283400d830 con 0x561752169fb0 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.962+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f283400b040 con 0x561752169fb0 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 10 client.15219 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 10 client.15219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 20 client.15219 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56175218d850 0x56175218fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 4 client.15219 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617520daff0 con 0x56175218d850 2024-04-24T22:42:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.963+0000 7f2833e00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56175218d850 0x56175218fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.964+0000 7f2833e00640 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56175218d850 0x56175218fc40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70db55ea5c5f914b server_cookie=9511e4d23f4640cd in_seq=0 out_seq=0 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.964+0000 7f2829600640 10 client.15219 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2829600640 1 -- 172.21.15.111:0/1285333852 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617520daff0 con 0x56175218d850 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2829600640 10 client.15219 handle_command_reply: tid=0 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 client.15219 shutdown 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 msgr2=0x5617521720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283400dee0 tx=0x7f2834005bd0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 shutdown_connections 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56175218d850 0x56175218fc40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561752175190 0x561752172b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56175216a8d0 0x561752172620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 --2- 172.21.15.111:0/1285333852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752169fb0 0x5617521720e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2829600640 0 client.15219 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 >> 172.21.15.111:0/1285333852 conn(0x5617521664c0 msgr2=0x56175216c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 shutdown_connections 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 1 -- 172.21.15.111:0/1285333852 wait complete. 2024-04-24T22:42:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.965+0000 7f2845a66740 20 client.15219 trim_cache size 0 max 16384 2024-04-24T22:42:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.981+0000 7f2845a66740 1 Processor -- start 2024-04-24T22:42:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 -- start start 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x561752171890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752195210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ad960 con 0x561752175190 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ab230 con 0x56175216a8d0 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2845a66740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1a9080 con 0x561752169fb0 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2840c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2840c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50916/0 (socket says 172.21.15.111:50916) 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2840c00640 1 -- 172.21.15.111:0/3042656854 learned_addr learned my addr 172.21.15.111:0/3042656854 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2833e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x561752171890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.982+0000 7f2840c00640 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 -- 172.21.15.111:0/3042656854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 msgr2=0x561752171890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x561752171890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 -- 172.21.15.111:0/3042656854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 msgr2=0x561752195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752195210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 -- 172.21.15.111:0/3042656854 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617521778f0 con 0x561752169fb0 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2833e00640 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x561752171890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f2840c00640 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2834009750 tx=0x7f2834005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae614a46313787a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f282be00640 1 -- 172.21.15.111:0/3042656854 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834017420 con 0x561752169fb0 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f282be00640 1 -- 172.21.15.111:0/3042656854 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283402c070 con 0x561752169fb0 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.983+0000 7f282be00640 1 -- 172.21.15.111:0/3042656854 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340052d0 con 0x561752169fb0 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- 172.21.15.111:0/3042656854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 msgr2=0x561752165860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2834009750 tx=0x7f2834005fd0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- 172.21.15.111:0/3042656854 shutdown_connections 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x561752171890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 --2- 172.21.15.111:0/3042656854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752165860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- 172.21.15.111:0/3042656854 >> 172.21.15.111:0/3042656854 conn(0x5617521664c0 msgr2=0x56175216c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- 172.21.15.111:0/3042656854 shutdown_connections 2024-04-24T22:42:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- 172.21.15.111:0/3042656854 wait complete. 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.984+0000 7f2845a66740 1 -- start start 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 Processor -- start 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x5617521812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752181a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1a96f0 con 0x561752175190 2024-04-24T22:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1aab60 con 0x56175216a8d0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1a9dc0 con 0x561752169fb0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2845a66740 10 client.0 _ll_register_callbacks cb 0x7f282c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2840c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2840c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50926/0 (socket says 172.21.15.111:50926) 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.985+0000 7f2840c00640 1 -- 172.21.15.111:0/346853710 learned_addr learned my addr 172.21.15.111:0/346853710 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752181a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2833e00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x5617521812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 -- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 msgr2=0x5617521812d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x5617521812d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 -- 172.21.15.111:0/346853710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 msgr2=0x561752181a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752181a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 -- 172.21.15.111:0/346853710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617521f2a80 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2840c00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f283400df60 tx=0x7f2834027040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ee5b65bbabf395 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2829600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400d830 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f2834003b80 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.986+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283402c070 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f283404a050 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 10 client.25330 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5617521f3c50 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283402c070 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f283400b040 con 0x561752169fb0 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 10 client.25330 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 10 client.25330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 20 client.25330 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5617521f4290 0x5617521f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 4 client.25330 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.987+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617520d5340 con 0x5617521f4290 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.988+0000 7f2833e00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5617521f4290 0x5617521f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.988+0000 7f2833e00640 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5617521f4290 0x5617521f66b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ade5a790c9375c8 server_cookie=a78815e4108cb8f1 in_seq=0 out_seq=0 2024-04-24T22:42:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:12.988+0000 7f2829600640 10 client.25330 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2829600640 1 -- 172.21.15.111:0/346853710 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617520d5340 con 0x5617521f4290 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2829600640 10 client.25330 handle_command_reply: tid=0 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 client.25330 shutdown 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 msgr2=0x561752185f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f283400df60 tx=0x7f2834027040 comp rx=0 tx=0).stop 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 shutdown_connections 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5617521f4290 0x5617521f66b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561752175190 0x561752181a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56175216a8d0 0x5617521812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 --2- 172.21.15.111:0/346853710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561752169fb0 0x561752185f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2829600640 0 client.25330 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.000+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 >> 172.21.15.111:0/346853710 conn(0x5617521664c0 msgr2=0x5617521720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 shutdown_connections 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2845a66740 1 -- 172.21.15.111:0/346853710 wait complete. 2024-04-24T22:42:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2845a66740 20 client.25330 trim_cache size 0 max 16384 2024-04-24T22:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 msgr2=0x7f28200a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 0x7f28200a5360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f282c0ccfe0 tx=0x7f282c0d34f0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 msgr2=0x7f283c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.001+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e9f0 tx=0x7f282c00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 shutdown_connections 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f283c109790 0x7f283c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f283c104140 0x7f283c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f28200a2ea0 0x7f28200a5360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 --2- 172.21.15.111:0/3651830168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f283c103780 0x7f283c19a060 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 >> 172.21.15.111:0/3651830168 conn(0x7f283c0ff250 msgr2=0x7f283c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 shutdown_connections 2024-04-24T22:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:13.002+0000 7f2841600640 1 -- 172.21.15.111:0/3651830168 wait complete. 2024-04-24T22:42:14.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:13 smithi171 ceph-mon[32337]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-24T22:42:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:13 smithi186 ceph-mon[32753]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-24T22:42:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:13 smithi111 ceph-mon[27746]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-24T22:42:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:15 smithi186 ceph-mon[32753]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-24T22:42:16.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:15 smithi171 ceph-mon[32337]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-24T22:42:16.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:15 smithi111 ceph-mon[27746]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-24T22:42:18.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:17 smithi171 ceph-mon[32337]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-24T22:42:18.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:17 smithi186 ceph-mon[32753]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-24T22:42:18.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:17 smithi111 ceph-mon[27746]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-24T22:42:20.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:19 smithi171 ceph-mon[32337]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-24T22:42:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:19 smithi186 ceph-mon[32753]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-24T22:42:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:19 smithi111 ceph-mon[27746]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-24T22:42:22.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:21 smithi171 ceph-mon[32337]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-24T22:42:22.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:21 smithi186 ceph-mon[32753]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-24T22:42:22.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:21 smithi111 ceph-mon[27746]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-24T22:42:24.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:24 smithi111 ceph-mon[27746]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T22:42:24.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:24 smithi171 ceph-mon[32337]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T22:42:24.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:24 smithi186 ceph-mon[32753]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T22:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 -- 172.21.15.111:0/3258342670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc104130 msgr2=0x7fdebc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 --2- 172.21.15.111:0/3258342670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc104130 0x7fdebc110640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009cd0 tx=0x7fdeb402a8f0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 -- 172.21.15.111:0/3258342670 shutdown_connections 2024-04-24T22:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 --2- 172.21.15.111:0/3258342670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc104130 0x7fdebc110640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 --2- 172.21.15.111:0/3258342670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc103770 0x7fdebc103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 --2- 172.21.15.111:0/3258342670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 -- 172.21.15.111:0/3258342670 >> 172.21.15.111:0/3258342670 conn(0x7fdebc0ff220 msgr2=0x7fdebc101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 -- 172.21.15.111:0/3258342670 shutdown_connections 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.145+0000 7fdec1400640 1 -- 172.21.15.111:0/3258342670 wait complete. 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.146+0000 7fdec1400640 1 Processor -- start 2024-04-24T22:42:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 -- start start 2024-04-24T22:42:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 0x7fdebc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc113ba0 con 0x7fdebc103770 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc113a20 con 0x7fdebc109780 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec1400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc113d20 con 0x7fdebc104130 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44202/0 (socket says 172.21.15.111:44202) 2024-04-24T22:42:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdebbe00640 1 -- 172.21.15.111:0/3965506346 learned_addr learned my addr 172.21.15.111:0/3965506346 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdebb400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 0x7fdebc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.147+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 -- 172.21.15.111:0/3965506346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 msgr2=0x7fdebc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 0x7fdebc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 -- 172.21.15.111:0/3965506346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 msgr2=0x7fdebc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 -- 172.21.15.111:0/3965506346 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc19eef0 con 0x7fdebc103770 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebb400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 0x7fdebc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdebbe00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c920 tx=0x7fdeac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdeb8c00640 1 -- 172.21.15.111:0/3965506346 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac007c70 con 0x7fdebc103770 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdeb8c00640 1 -- 172.21.15.111:0/3965506346 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeac007e10 con 0x7fdebc103770 2024-04-24T22:42:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc19f1e0 con 0x7fdebc103770 2024-04-24T22:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.148+0000 7fdeb8c00640 1 -- 172.21.15.111:0/3965506346 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac018890 con 0x7fdebc103770 2024-04-24T22:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.149+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc1a6a70 con 0x7fdebc103770 2024-04-24T22:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdeb8c00640 1 -- 172.21.15.111:0/3965506346 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7fdeac018a70 con 0x7fdebc103770 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 Processor -- start 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 -- start start 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a558d0 0x557b88a5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 0x557b88a62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b889b8ab0 con 0x557b88a60190 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b88961640 con 0x557b88a558d0 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.152+0000 7fdec5863740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557b889b4a70 con 0x557b88a54fb0 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdebb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 0x557b88a62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:49840/0 (socket says 172.21.15.111:49840) 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a558d0 0x557b88a5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 -- 172.21.15.111:0/318941752 learned_addr learned my addr 172.21.15.111:0/318941752 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 -- 172.21.15.111:0/318941752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a558d0 msgr2=0x557b88a5fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a558d0 0x557b88a5fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 -- 172.21.15.111:0/318941752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 msgr2=0x557b88a62580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 0x557b88a62580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdec0a00640 1 -- 172.21.15.111:0/318941752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b886cd210 con 0x557b88a54fb0 2024-04-24T22:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdebb400640 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 0x557b88a62580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.153+0000 7fdeb8c00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 0x7fdea00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdec0a00640 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4035000 tx=0x7fdeb4030560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70be1b49deec7d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdeb8c00640 1 -- 172.21.15.111:0/3965506346 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fdeac0cf800 con 0x7fdebc103770 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdebbe00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 0x7fdea00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdeb1200640 1 -- 172.21.15.111:0/318941752 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4003b70 con 0x557b88a54fb0 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdeb1200640 1 -- 172.21.15.111:0/318941752 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4036040 con 0x557b88a54fb0 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.154+0000 7fdeb1200640 1 -- 172.21.15.111:0/318941752 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4004580 con 0x557b88a54fb0 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- 172.21.15.111:0/318941752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 msgr2=0x557b88a55390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4035000 tx=0x7fdeb4030560 comp rx=0 tx=0).stop 2024-04-24T22:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdebbe00640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 0x7fdea00a53e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0ccda0 tx=0x7fdeac0d5530 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- 172.21.15.111:0/318941752 shutdown_connections 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a60190 0x557b88a62580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a558d0 0x557b88a5fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 --2- 172.21.15.111:0/318941752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a54fb0 0x557b88a55390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- 172.21.15.111:0/318941752 >> 172.21.15.111:0/318941752 conn(0x557b88a514c0 msgr2=0x557b88a518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- 172.21.15.111:0/318941752 shutdown_connections 2024-04-24T22:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- 172.21.15.111:0/318941752 wait complete. 2024-04-24T22:42:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 -- start start 2024-04-24T22:42:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.155+0000 7fdec5863740 1 Processor -- start 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a5d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a5db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b88a62d60 con 0x557b88a54fb0 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b88974990 con 0x557b88a60190 2024-04-24T22:42:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557b88a62ee0 con 0x557b88a558d0 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec5863740 10 client.0 _ll_register_callbacks cb 0x557b88976950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a5d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44206/0 (socket says 172.21.15.111:44206) 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec0a00640 1 -- 172.21.15.111:0/3657022650 learned_addr learned my addr 172.21.15.111:0/3657022650 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdebb400640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a5d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.156+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a5db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 -- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 msgr2=0x557b88a5db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a5db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 -- 172.21.15.111:0/3657022650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 msgr2=0x557b88a5d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a5d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 -- 172.21.15.111:0/3657022650 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b88799620 con 0x557b88a558d0 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdebb400640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a5d650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x557b88a52650 tx=0x7fdea8011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981b0a5e4e21c262 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdea5600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea800fdc0 con 0x557b88a558d0 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4036040 con 0x557b88a558d0 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.157+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fdea8005020 con 0x557b88a558d0 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdea8005220 con 0x557b88a558d0 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 10 client.25339 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557b88a6ecc0 con 0x557b88a558d0 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4036040 con 0x557b88a558d0 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdea8018070 con 0x557b88a558d0 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 10 client.25339 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 10 client.25339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 20 client.25339 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88a70130 0x557b88a72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 4 client.25339 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:42:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.158+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b889c5ff0 con 0x557b88a70130 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.159+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88a70130 0x557b88a72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.159+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88a70130 0x557b88a72560 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23f18538fb32ac9 server_cookie=d880e8da8c6e8d3a in_seq=0 out_seq=0 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.159+0000 7fdea5600640 10 client.25339 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdea5600640 1 -- 172.21.15.111:0/3657022650 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b889c5ff0 con 0x557b88a70130 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdea5600640 10 client.25339 handle_command_reply: tid=0 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 client.25339 shutdown 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 msgr2=0x557b88a5d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a5d650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x557b88a52650 tx=0x7fdea8011c90 comp rx=0 tx=0).stop 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 shutdown_connections 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88a70130 0x557b88a72560 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a5d650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 --2- 172.21.15.111:0/3657022650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a5d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdea5600640 0 client.25339 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 >> 172.21.15.111:0/3657022650 conn(0x557b88a514c0 msgr2=0x557b88a571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 shutdown_connections 2024-04-24T22:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.160+0000 7fdec5863740 1 -- 172.21.15.111:0/3657022650 wait complete. 2024-04-24T22:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.161+0000 7fdec5863740 20 client.25339 trim_cache size 0 max 16384 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.176+0000 7fdec5863740 1 Processor -- start 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 -- start start 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 0x557b88a77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a60190 0x557b88a783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1b1c90 con 0x557b88a558d0 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1af560 con 0x557b88a54fb0 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec5863740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ad3b0 con 0x557b88a60190 2024-04-24T22:42:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdebb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 0x557b88a77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33542/0 (socket says 172.21.15.111:33542) 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec0a00640 1 -- 172.21.15.111:0/1427775043 learned_addr learned my addr 172.21.15.111:0/1427775043 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.177+0000 7fdec0a00640 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a60190 0x557b88a783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 -- 172.21.15.111:0/1427775043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a60190 msgr2=0x557b88a783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a60190 0x557b88a783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 -- 172.21.15.111:0/1427775043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 msgr2=0x557b88a50da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 -- 172.21.15.111:0/1427775043 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b88a512e0 con 0x557b88a558d0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdec0a00640 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdebb400640 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 0x557b88a77e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdea8019700 tx=0x7fdea8019990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1e16717c315e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdea7e00640 1 -- 172.21.15.111:0/1427775043 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea802f070 con 0x557b88a558d0 2024-04-24T22:42:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdea7e00640 1 -- 172.21.15.111:0/1427775043 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea80055c0 con 0x557b88a558d0 2024-04-24T22:42:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.178+0000 7fdea7e00640 1 -- 172.21.15.111:0/1427775043 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8026040 con 0x557b88a558d0 2024-04-24T22:42:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 -- 172.21.15.111:0/1427775043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 msgr2=0x557b88a77e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 0x557b88a77e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdea8019700 tx=0x7fdea8019990 comp rx=0 tx=0).stop 2024-04-24T22:42:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 -- 172.21.15.111:0/1427775043 shutdown_connections 2024-04-24T22:42:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a60190 0x557b88a783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a558d0 0x557b88a77e70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 --2- 172.21.15.111:0/1427775043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a54fb0 0x557b88a50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 -- 172.21.15.111:0/1427775043 >> 172.21.15.111:0/1427775043 conn(0x557b88a514c0 msgr2=0x557b88a56960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 -- 172.21.15.111:0/1427775043 shutdown_connections 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.179+0000 7fdec5863740 1 -- 172.21.15.111:0/1427775043 wait complete. 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 -- start start 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 Processor -- start 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a65640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ada20 con 0x557b88a54fb0 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1aee90 con 0x557b88a60190 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ae0f0 con 0x557b88a558d0 2024-04-24T22:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.180+0000 7fdec5863740 10 client.0 _ll_register_callbacks cb 0x7fdea80087c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44222/0 (socket says 172.21.15.111:44222) 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdec0a00640 1 -- 172.21.15.111:0/3665431197 learned_addr learned my addr 172.21.15.111:0/3665431197 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a65100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a65640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 -- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 msgr2=0x557b88a65640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a65640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 -- 172.21.15.111:0/3665431197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 msgr2=0x557b88a68390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 -- 172.21.15.111:0/3665431197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b88a65cb0 con 0x557b88a558d0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdebb400640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a65100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdea8019590 tx=0x7fdea8005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85b34ebb6757ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdea5600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.181+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8038060 con 0x557b88a558d0 2024-04-24T22:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea80055c0 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fdea800fbc0 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdea8026070 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 10 client.25345 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557b88ad7570 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea80055c0 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdea8032030 con 0x557b88a558d0 2024-04-24T22:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 10 client.25345 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.182+0000 7fdec5863740 20 client.25345 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88ad7b70 0x557b88ad9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdec5863740 4 client.25345 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b889c0340 con 0x557b88ad7b70 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88ad7b70 0x557b88ad9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdec0a00640 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88ad7b70 0x557b88ad9f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75775ea009eefafc server_cookie=d176ac54467fb06 in_seq=0 out_seq=0 2024-04-24T22:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.183+0000 7fdea5600640 10 client.25345 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.187+0000 7fdea5600640 1 -- 172.21.15.111:0/3665431197 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b889c0340 con 0x557b88ad7b70 2024-04-24T22:42:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.187+0000 7fdea5600640 10 client.25345 handle_command_reply: tid=0 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.187+0000 7fdec5863740 1 client.25345 shutdown 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.187+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 msgr2=0x557b88a65100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.187+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a65100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdea8019590 tx=0x7fdea8005770 comp rx=0 tx=0).stop 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 shutdown_connections 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557b88ad7b70 0x557b88ad9f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b88a60190 0x557b88a65640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b88a558d0 0x557b88a65100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 --2- 172.21.15.111:0/3665431197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b88a54fb0 0x557b88a68390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdea5600640 0 client.25345 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 >> 172.21.15.111:0/3665431197 conn(0x557b88a514c0 msgr2=0x557b88a571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 shutdown_connections 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 1 -- 172.21.15.111:0/3665431197 wait complete. 2024-04-24T22:42:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec5863740 20 client.25345 trim_cache size 0 max 16384 2024-04-24T22:42:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:42:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:42:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:42:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 msgr2=0x7fdea00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 0x7fdea00a53e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0ccda0 tx=0x7fdeac0d5530 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 msgr2=0x7fdebc199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.188+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c920 tx=0x7fdeac00cde0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 shutdown_connections 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdebc109780 0x7fdebc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdea00a2f20 0x7fdea00a53e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdebc104130 0x7fdebc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 --2- 172.21.15.111:0/3965506346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc103770 0x7fdebc199f00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 >> 172.21.15.111:0/3965506346 conn(0x7fdebc0ff220 msgr2=0x7fdebc0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 shutdown_connections 2024-04-24T22:42:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:26.189+0000 7fdec1400640 1 -- 172.21.15.111:0/3965506346 wait complete. 2024-04-24T22:42:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:26 smithi111 ceph-mon[27746]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-24T22:42:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:26 smithi171 ceph-mon[32337]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-24T22:42:26.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:26 smithi186 ceph-mon[32753]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-24T22:42:28.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:28 smithi111 ceph-mon[27746]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T22:42:28.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:28 smithi171 ceph-mon[32337]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T22:42:28.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:28 smithi186 ceph-mon[32753]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T22:42:30.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:30 smithi111 ceph-mon[27746]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-24T22:42:30.408 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:30 smithi171 ceph-mon[32337]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-24T22:42:30.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:30 smithi186 ceph-mon[32753]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-24T22:42:32.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:32 smithi111 ceph-mon[27746]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:32.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:32 smithi171 ceph-mon[32337]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:32.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:32 smithi186 ceph-mon[32753]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:34.302 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:34 smithi186 ceph-mon[32753]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:42:34.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:34 smithi171 ceph-mon[32337]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:42:34.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:34 smithi111 ceph-mon[27746]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:42:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:36 smithi171 ceph-mon[32337]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:42:36.495 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:36 smithi186 ceph-mon[32753]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:42:36.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:36 smithi111 ceph-mon[27746]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T22:42:38.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:38 smithi171 ceph-mon[32337]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:38.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:38 smithi186 ceph-mon[32753]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:38 smithi111 ceph-mon[27746]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 -- 172.21.15.111:0/1341162319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 msgr2=0x7f3784109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 --2- 172.21.15.111:0/1341162319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f3784109d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3774009900 tx=0x7f377402a410 comp rx=0 tx=0).stop 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 -- 172.21.15.111:0/1341162319 shutdown_connections 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 --2- 172.21.15.111:0/1341162319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37841042f0 0x7f3784110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 --2- 172.21.15.111:0/1341162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3784103930 0x7f3784103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 --2- 172.21.15.111:0/1341162319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f3784109d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 -- 172.21.15.111:0/1341162319 >> 172.21.15.111:0/1341162319 conn(0x7f37840ff440 msgr2=0x7f3784101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 -- 172.21.15.111:0/1341162319 shutdown_connections 2024-04-24T22:42:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.324+0000 7f3788a00640 1 -- 172.21.15.111:0/1341162319 wait complete. 2024-04-24T22:42:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 Processor -- start 2024-04-24T22:42:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 -- start start 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 0x7f378419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f378419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3784113da0 con 0x7f3784103930 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3784113c20 con 0x7f3784109940 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.326+0000 7f3788a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3784113f20 con 0x7f37841042f0 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57670/0 (socket says 172.21.15.111:57670) 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 -- 172.21.15.111:0/4073812918 learned_addr learned my addr 172.21.15.111:0/4073812918 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781400640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 0x7f378419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3782800640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f378419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 -- 172.21.15.111:0/4073812918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 msgr2=0x7f378419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 0x7f378419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 -- 172.21.15.111:0/4073812918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 msgr2=0x7f378419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f378419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781e00640 1 -- 172.21.15.111:0/4073812918 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419f170 con 0x7f3784103930 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3781400640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 0x7f378419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.327+0000 7f3782800640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f378419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f3781e00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f377402aa00 tx=0x7f3774003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f377aa00640 1 -- 172.21.15.111:0/4073812918 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377403f070 con 0x7f3784103930 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f377aa00640 1 -- 172.21.15.111:0/4073812918 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3774031c30 con 0x7f3784103930 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419f400 con 0x7f3784103930 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f377aa00640 1 -- 172.21.15.111:0/4073812918 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774035030 con 0x7f3784103930 2024-04-24T22:42:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.328+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a6cf0 con 0x7f3784103930 2024-04-24T22:42:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 Processor -- start 2024-04-24T22:42:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 -- start start 2024-04-24T22:42:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a26384fab0 con 0x55a2638ec720 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a2637f8640 con 0x55a2638ebe00 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f378b7fe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a26384ba70 con 0x55a2638f6fe0 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f3782800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f3782800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:60748/0 (socket says 172.21.15.111:60748) 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.332+0000 7f3782800640 1 -- 172.21.15.111:0/1635869389 learned_addr learned my addr 172.21.15.111:0/1635869389 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3781400640 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 -- 172.21.15.111:0/1635869389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 msgr2=0x55a2638f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 -- 172.21.15.111:0/1635869389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 msgr2=0x55a2638f6aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f6aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 -- 172.21.15.111:0/1635869389 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a263564210 con 0x55a2638ebe00 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f377aa00640 1 -- 172.21.15.111:0/4073812918 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f3774042070 con 0x7f3784103930 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f3782800640 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f377c011790 tx=0x7f377c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed367bcb7353919 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f376f000640 1 -- 172.21.15.111:0/1635869389 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c004d20 con 0x55a2638ebe00 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f376f000640 1 -- 172.21.15.111:0/1635869389 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c00f9a0 con 0x55a2638ebe00 2024-04-24T22:42:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.333+0000 7f376f000640 1 -- 172.21.15.111:0/1635869389 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c004550 con 0x55a2638ebe00 2024-04-24T22:42:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 -- 172.21.15.111:0/1635869389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 msgr2=0x55a2638ec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f377c011790 tx=0x7f377c011c50 comp rx=0 tx=0).stop 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 -- 172.21.15.111:0/1635869389 shutdown_connections 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1635869389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638ec1e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 -- 172.21.15.111:0/1635869389 >> 172.21.15.111:0/1635869389 conn(0x55a2638e8310 msgr2=0x55a2638e86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 -- 172.21.15.111:0/1635869389 shutdown_connections 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.334+0000 7f378b7fe740 1 -- 172.21.15.111:0/1635869389 wait complete. 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.335+0000 7f377aa00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 0x7f37680a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.335+0000 7f377aa00640 1 -- 172.21.15.111:0/4073812918 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f37740efcb0 con 0x7f3784103930 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.335+0000 7f378b7fe740 1 -- start start 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.335+0000 7f3781400640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 0x7f37680a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.335+0000 7f378b7fe740 1 Processor -- start 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3781400640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 0x7f37680a5320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x55a2638e9730 tx=0x7f3770005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ebe00 0x55a2638f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 0x55a2638f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a2638f9c10 con 0x55a2638ec720 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a26380b990 con 0x55a2638f6fe0 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2638f9d90 con 0x55a2638ebe00 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f378b7fe740 10 client.0 _ll_register_callbacks cb 0x55a26380d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3781400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3781400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57676/0 (socket says 172.21.15.111:57676) 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3781400640 1 -- 172.21.15.111:0/1473006032 learned_addr learned my addr 172.21.15.111:0/1473006032 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3782800640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 0x55a2638f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.336+0000 7f3781e00640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ebe00 0x55a2638f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 -- 172.21.15.111:0/1473006032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ebe00 msgr2=0x55a2638f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ebe00 0x55a2638f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 -- 172.21.15.111:0/1473006032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 msgr2=0x55a2638f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 -- 172.21.15.111:0/1473006032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a263630620 con 0x55a2638f6fe0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3782800640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 0x55a2638f4a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f377c005c10 tx=0x7f377c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8312270bad3c7dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3763400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c028020 con 0x55a2638f6fe0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c00f9a0 con 0x55a2638f6fe0 2024-04-24T22:42:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.337+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f377c005410 con 0x55a2638f6fe0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f377c033050 con 0x55a2638f6fe0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f378b7fe740 10 client.25340 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a263905bb0 con 0x55a2638f6fe0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c00f9a0 con 0x55a2638f6fe0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f377c018070 con 0x55a2638f6fe0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f378b7fe740 10 client.25340 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.338+0000 7f378b7fe740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f378b7fe740 20 client.25340 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a263906fc0 0x55a2639093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f378b7fe740 4 client.25340 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a26385cff0 con 0x55a263906fc0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f3781e00640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a263906fc0 0x55a2639093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f3781e00640 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a263906fc0 0x55a2639093b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8416966bbedf10 server_cookie=ffe6d3ed1732345 in_seq=0 out_seq=0 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.339+0000 7f3763400640 10 client.25340 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f3763400640 1 -- 172.21.15.111:0/1473006032 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a26385cff0 con 0x55a263906fc0 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f3763400640 10 client.25340 handle_command_reply: tid=0 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 client.25340 shutdown 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 msgr2=0x55a2638f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 0x55a2638f4a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f377c005c10 tx=0x7f377c003ce0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 shutdown_connections 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a263906fc0 0x55a2639093b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638f6fe0 0x55a2638f4a70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ec720 0x55a2638f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 --2- 172.21.15.111:0/1473006032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ebe00 0x55a2638f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.340+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 >> 172.21.15.111:0/1473006032 conn(0x55a2638e8310 msgr2=0x55a2638ee090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.341+0000 7f3763400640 0 client.25340 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.341+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 shutdown_connections 2024-04-24T22:42:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.341+0000 7f378b7fe740 1 -- 172.21.15.111:0/1473006032 wait complete. 2024-04-24T22:42:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.341+0000 7f378b7fe740 20 client.25340 trim_cache size 0 max 16384 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 Processor -- start 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 -- start start 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ec720 0x55a2638ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f37841b1ed0 con 0x55a2638f6fe0 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37841af7a0 con 0x55a2638ebe00 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f378b7fe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f37841ad5f0 con 0x55a2638ec720 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f3782800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f3781e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f3781400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ec720 0x55a2638ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f3782800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57688/0 (socket says 172.21.15.111:57688) 2024-04-24T22:42:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.357+0000 7f3782800640 1 -- 172.21.15.111:0/3220499033 learned_addr learned my addr 172.21.15.111:0/3220499033 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 -- 172.21.15.111:0/3220499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ec720 msgr2=0x55a2638ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ec720 0x55a2638ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 -- 172.21.15.111:0/3220499033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 msgr2=0x55a2638e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 -- 172.21.15.111:0/3220499033 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2638e8140 con 0x55a2638f6fe0 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3781e00640 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f3782800640 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f377c011c90 tx=0x7f377c00ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e47247ee8865d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f376e600640 1 -- 172.21.15.111:0/3220499033 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c028030 con 0x55a2638f6fe0 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f376e600640 1 -- 172.21.15.111:0/3220499033 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c01f040 con 0x55a2638f6fe0 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.358+0000 7f376e600640 1 -- 172.21.15.111:0/3220499033 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c024660 con 0x55a2638f6fe0 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 -- 172.21.15.111:0/3220499033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 msgr2=0x55a26390eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f377c011c90 tx=0x7f377c00ff50 comp rx=0 tx=0).stop 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 -- 172.21.15.111:0/3220499033 shutdown_connections 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638f6fe0 0x55a26390eb20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638ec720 0x55a2638ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 --2- 172.21.15.111:0/3220499033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ebe00 0x55a2638e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 -- 172.21.15.111:0/3220499033 >> 172.21.15.111:0/3220499033 conn(0x55a2638e8310 msgr2=0x55a2638f4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 -- 172.21.15.111:0/3220499033 shutdown_connections 2024-04-24T22:42:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.359+0000 7f378b7fe740 1 -- 172.21.15.111:0/3220499033 wait complete. 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 -- start start 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 Processor -- start 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ec720 0x55a2638f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f37841adc60 con 0x55a2638ebe00 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37841af0d0 con 0x55a2638ec720 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.360+0000 7f378b7fe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f37841ae330 con 0x55a2638f6fe0 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f378b7fe740 10 client.0 _ll_register_callbacks cb 0x7f3770009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57690/0 (socket says 172.21.15.111:57690) 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 -- 172.21.15.111:0/840727147 learned_addr learned my addr 172.21.15.111:0/840727147 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781400640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ec720 0x55a2638f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3782800640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 -- 172.21.15.111:0/840727147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 msgr2=0x55a2638f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 -- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ec720 msgr2=0x55a2638f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ec720 0x55a2638f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 -- 172.21.15.111:0/840727147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2638f1860 con 0x55a2638ebe00 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3781e00640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37740b8a40 tx=0x7f37740dc170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef9dffef1de03fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.361+0000 7f3763400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774049040 con 0x55a2638ebe00 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c01f040 con 0x55a2638ebe00 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f37740b6be0 con 0x55a2638ebe00 2024-04-24T22:42:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f37740bc070 con 0x55a2638ebe00 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a26396d4a0 con 0x55a2638ebe00 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c01f040 con 0x55a2638ebe00 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f37740be040 con 0x55a2638ebe00 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 10 client.15258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 20 client.15258 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a26396db70 0x55a26396ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 4 client.15258 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.362+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a263857340 con 0x55a26396db70 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.363+0000 7f3781400640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a26396db70 0x55a26396ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.363+0000 7f3781400640 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a26396db70 0x55a26396ff90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586fb5f23a6710e9 server_cookie=348b8a1b18027600 in_seq=0 out_seq=0 2024-04-24T22:42:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.363+0000 7f3763400640 10 client.15258 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.370+0000 7f3763400640 1 -- 172.21.15.111:0/840727147 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a263857340 con 0x55a26396db70 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.370+0000 7f3763400640 10 client.15258 handle_command_reply: tid=0 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.370+0000 7f378b7fe740 1 client.15258 shutdown 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 msgr2=0x55a2638ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37740b8a40 tx=0x7f37740dc170 comp rx=0 tx=0).stop 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 shutdown_connections 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a26396db70 0x55a26396ff90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2638f6fe0 0x55a2638f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2638ec720 0x55a2638f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 --2- 172.21.15.111:0/840727147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2638ebe00 0x55a2638ff4c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f3763400640 0 client.15258 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 >> 172.21.15.111:0/840727147 conn(0x55a2638e8310 msgr2=0x55a2638f5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 shutdown_connections 2024-04-24T22:42:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 1 -- 172.21.15.111:0/840727147 wait complete. 2024-04-24T22:42:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.371+0000 7f378b7fe740 20 client.15258 trim_cache size 0 max 16384 2024-04-24T22:42:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:42:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:42:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:42:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 msgr2=0x7f37680a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 0x7f37680a5320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x55a2638e9730 tx=0x7f3770005f90 comp rx=0 tx=0).stop 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 msgr2=0x7f378419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f377402aa00 tx=0x7f3774003c10 comp rx=0 tx=0).stop 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 shutdown_connections 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3784109940 0x7f378419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f37680a2e60 0x7f37680a5320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37841042f0 0x7f378419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 --2- 172.21.15.111:0/4073812918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3784103930 0x7f378419a180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 >> 172.21.15.111:0/4073812918 conn(0x7f37840ff440 msgr2=0x7f37840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 shutdown_connections 2024-04-24T22:42:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:39.372+0000 7f3788a00640 1 -- 172.21.15.111:0/4073812918 wait complete. 2024-04-24T22:42:40.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:40 smithi171 ceph-mon[32337]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-24T22:42:40.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:40 smithi186 ceph-mon[32753]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-24T22:42:40.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:40 smithi111 ceph-mon[27746]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-24T22:42:42.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:42 smithi171 ceph-mon[32337]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-24T22:42:42.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:42 smithi186 ceph-mon[32753]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-24T22:42:42.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:42 smithi111 ceph-mon[27746]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-24T22:42:44.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:44 smithi171 ceph-mon[32337]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-24T22:42:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:44 smithi186 ceph-mon[32753]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-24T22:42:44.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:44 smithi111 ceph-mon[27746]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-24T22:42:46.410 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:46 smithi171 ceph-mon[32337]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:46.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:46 smithi186 ceph-mon[32753]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:46.595 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:46 smithi111 ceph-mon[27746]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:48.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:48 smithi171 ceph-mon[32337]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:48 smithi186 ceph-mon[32753]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:48.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:48 smithi111 ceph-mon[27746]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-24T22:42:50.416 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:50 smithi171 ceph-mon[32337]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-24T22:42:50.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:50 smithi186 ceph-mon[32753]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-24T22:42:50.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:50 smithi111 ceph-mon[27746]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-24T22:42:52.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:52 smithi171 ceph-mon[32337]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T22:42:52.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:52 smithi186 ceph-mon[32753]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 -- 172.21.15.111:0/4274604767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 msgr2=0x7f39a4110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 --2- 172.21.15.111:0/4274604767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a4110210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f399c009ca0 tx=0x7f399c02a950 comp rx=0 tx=0).stop 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 -- 172.21.15.111:0/4274604767 shutdown_connections 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 --2- 172.21.15.111:0/4274604767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a4110210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 --2- 172.21.15.111:0/4274604767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 --2- 172.21.15.111:0/4274604767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 -- 172.21.15.111:0/4274604767 >> 172.21.15.111:0/4274604767 conn(0x7f39a40fd090 msgr2=0x7f39a40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 -- 172.21.15.111:0/4274604767 shutdown_connections 2024-04-24T22:42:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.511+0000 7f39a3200640 1 -- 172.21.15.111:0/4274604767 wait complete. 2024-04-24T22:42:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.513+0000 7f39a3200640 1 Processor -- start 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.513+0000 7f39a3200640 1 -- start start 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.513+0000 7f39a3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.513+0000 7f39a3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.513+0000 7f39a3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39a4113bb0 con 0x7f39a4073b10 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39a4113a30 con 0x7f39a410de00 2024-04-24T22:42:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a3200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f39a4113d30 con 0x7f39a4074430 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59240/0 (socket says 172.21.15.111:59240) 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1e00640 1 -- 172.21.15.111:0/1914224211 learned_addr learned my addr 172.21.15.111:0/1914224211 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a2800640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 -- 172.21.15.111:0/1914224211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 msgr2=0x7f39a419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 -- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 msgr2=0x7f39a419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1400640 1 -- 172.21.15.111:0/1914224211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a419f060 con 0x7f39a4074430 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.514+0000 7f39a1e00640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f39a1400640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a419a650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9f0 tx=0x7f399000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f399aa00640 1 -- 172.21.15.111:0/1914224211 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000ccc0 con 0x7f39a4074430 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a419f350 con 0x7f39a4074430 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f399aa00640 1 -- 172.21.15.111:0/1914224211 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3990010910 con 0x7f39a4074430 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f399aa00640 1 -- 172.21.15.111:0/1914224211 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990018960 con 0x7f39a4074430 2024-04-24T22:42:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.515+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a41a6be0 con 0x7f39a4074430 2024-04-24T22:42:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.519+0000 7f39abd96740 1 Processor -- start 2024-04-24T22:42:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.519+0000 7f39abd96740 1 -- start start 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219723c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219724190 0x561219726580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56121967cab0 con 0x561219724190 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561219625640 con 0x561219718fb0 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39abd96740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561219678a70 con 0x5612197198d0 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f399aa00640 1 -- 172.21.15.111:0/1914224211 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f3990010ac0 con 0x7f39a4074430 2024-04-24T22:42:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39a2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39a2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:60874/0 (socket says 172.21.15.111:60874) 2024-04-24T22:42:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39a2800640 1 -- 172.21.15.111:0/1113666398 learned_addr learned my addr 172.21.15.111:0/1113666398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39a1400640 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219724190 0x561219726580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.520+0000 7f39a1e00640 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219723c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 -- 172.21.15.111:0/1113666398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 msgr2=0x561219723c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219723c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 -- 172.21.15.111:0/1113666398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219724190 msgr2=0x561219726580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219724190 0x561219726580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 -- 172.21.15.111:0/1113666398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561219391210 con 0x561219718fb0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f39a2800640 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39a4074a60 tx=0x7f399c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c95ace83ad96b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f399aa00640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 0x7f39880a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.521+0000 7f398f400640 1 -- 172.21.15.111:0/1113666398 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c048070 con 0x561219718fb0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f398f400640 1 -- 172.21.15.111:0/1113666398 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399c003e50 con 0x561219718fb0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f398f400640 1 -- 172.21.15.111:0/1113666398 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c034040 con 0x561219718fb0 2024-04-24T22:42:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f399aa00640 1 -- 172.21.15.111:0/1914224211 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f39900cfa10 con 0x7f39a4074430 2024-04-24T22:42:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 -- 172.21.15.111:0/1113666398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 msgr2=0x561219719390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39a4074a60 tx=0x7f399c005f80 comp rx=0 tx=0).stop 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 -- 172.21.15.111:0/1113666398 shutdown_connections 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219724190 0x561219726580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39a1e00640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 0x7f39880a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219723c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 --2- 172.21.15.111:0/1113666398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x561219719390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.522+0000 7f39abd96740 1 -- 172.21.15.111:0/1113666398 >> 172.21.15.111:0/1113666398 conn(0x5612197154c0 msgr2=0x5612197158a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.523+0000 7f39abd96740 1 -- 172.21.15.111:0/1113666398 shutdown_connections 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.523+0000 7f39abd96740 1 -- 172.21.15.111:0/1113666398 wait complete. 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.523+0000 7f39a1e00640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 0x7f39880a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3994009620 tx=0x7f3994005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.523+0000 7f39abd96740 1 -- start start 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 Processor -- start 2024-04-24T22:42:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x561219721640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219721b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561219726d60 con 0x5612197198d0 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561219638990 con 0x561219724190 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561219726ee0 con 0x561219718fb0 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.524+0000 7f39abd96740 10 client.0 _ll_register_callbacks cb 0x56121963a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x561219721640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:54752/0 (socket says 172.21.15.111:54752) 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 -- 172.21.15.111:0/4176727227 learned_addr learned my addr 172.21.15.111:0/4176727227 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a2800640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219721b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 -- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 msgr2=0x561219721b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219721b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 -- 172.21.15.111:0/4176727227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 msgr2=0x561219721640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x561219721640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 -- 172.21.15.111:0/4176727227 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121945d620 con 0x561219718fb0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f399400f880 tx=0x7f3994004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b93ed716b457e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.525+0000 7f3983e00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39940119c0 con 0x561219718fb0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f3994011b60 con 0x561219718fb0 2024-04-24T22:42:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f399c003e50 con 0x561219718fb0 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3994011da0 con 0x561219718fb0 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f39abd96740 10 client.25375 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.526+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561219732cc0 con 0x561219718fb0 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399c003e50 con 0x561219718fb0 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3994003cd0 con 0x561219718fb0 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 10 client.25375 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 20 client.25375 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5612197341d0 0x5612197365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 4 client.25375 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:42:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.527+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561219689ff0 con 0x5612197341d0 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.528+0000 7f39a1400640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5612197341d0 0x5612197365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.528+0000 7f39a1400640 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5612197341d0 0x5612197365c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b0f0c51a5ac0df server_cookie=6ccaefc24b5b4bea in_seq=0 out_seq=0 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.528+0000 7f3983e00640 10 client.25375 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.529+0000 7f3983e00640 1 -- 172.21.15.111:0/4176727227 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561219689ff0 con 0x5612197341d0 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.529+0000 7f3983e00640 10 client.25375 handle_command_reply: tid=0 2024-04-24T22:42:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.529+0000 7f39abd96740 1 client.25375 shutdown 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.529+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 msgr2=0x561219721100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.529+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f399400f880 tx=0x7f3994004730 comp rx=0 tx=0).stop 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 shutdown_connections 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5612197341d0 0x5612197365c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219721b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x561219721640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 --2- 172.21.15.111:0/4176727227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219718fb0 0x561219721100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f3983e00640 0 client.25375 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 >> 172.21.15.111:0/4176727227 conn(0x5612197154c0 msgr2=0x56121971b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 shutdown_connections 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 1 -- 172.21.15.111:0/4176727227 wait complete. 2024-04-24T22:42:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.530+0000 7f39abd96740 20 client.25375 trim_cache size 0 max 16384 2024-04-24T22:42:52.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:52 smithi111 ceph-mon[27746]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 Processor -- start 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 -- start start 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x56121972fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219724190 0x56121971a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41b1c70 con 0x5612197198d0 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41af540 con 0x561219718fb0 2024-04-24T22:42:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39abd96740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41ad390 con 0x561219724190 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39a1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x56121972fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.560+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:60898/0 (socket says 172.21.15.111:60898) 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x56121972fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59266/0 (socket says 172.21.15.111:59266) 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 -- 172.21.15.111:0/2218909189 learned_addr learned my addr 172.21.15.111:0/2218909189 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a2800640 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219724190 0x56121971a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 -- 172.21.15.111:0/2218909189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219724190 msgr2=0x56121971a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219724190 0x56121971a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 -- 172.21.15.111:0/2218909189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 msgr2=0x56121972fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x56121972fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 -- 172.21.15.111:0/2218909189 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612197200d0 con 0x561219718fb0 2024-04-24T22:42:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.561+0000 7f39a1e00640 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3994013010 tx=0x7f3994029ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8d44b8e900ea7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.562+0000 7f398ea00640 1 -- 172.21.15.111:0/2218909189 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994029dc0 con 0x561219718fb0 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.562+0000 7f398ea00640 1 -- 172.21.15.111:0/2218909189 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3994004030 con 0x561219718fb0 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.562+0000 7f398ea00640 1 -- 172.21.15.111:0/2218909189 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994005280 con 0x561219718fb0 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- 172.21.15.111:0/2218909189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 msgr2=0x5612197238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3994013010 tx=0x7f3994029ab0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- 172.21.15.111:0/2218909189 shutdown_connections 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561219724190 0x56121971a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612197198d0 0x56121972fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 --2- 172.21.15.111:0/2218909189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219718fb0 0x5612197238d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- 172.21.15.111:0/2218909189 >> 172.21.15.111:0/2218909189 conn(0x5612197154c0 msgr2=0x5612197231c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- 172.21.15.111:0/2218909189 shutdown_connections 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- 172.21.15.111:0/2218909189 wait complete. 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.563+0000 7f39abd96740 1 -- start start 2024-04-24T22:42:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 Processor -- start 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219729100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219729640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41ada00 con 0x561219718fb0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41aee70 con 0x561219724190 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41ae0d0 con 0x5612197198d0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39abd96740 10 client.0 _ll_register_callbacks cb 0x7f399400f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.564+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59278/0 (socket says 172.21.15.111:59278) 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 -- 172.21.15.111:0/4214729152 learned_addr learned my addr 172.21.15.111:0/4214729152 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a2800640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219729640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1400640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219729100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 -- 172.21.15.111:0/4214729152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 msgr2=0x561219729100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219729100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 -- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 msgr2=0x561219729640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219729640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 -- 172.21.15.111:0/4214729152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561219736930 con 0x561219718fb0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f39a1e00640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3994029580 tx=0x7f3994029bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20253f80da43c0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.565+0000 7f3983e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994011a20 con 0x561219718fb0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f3994005020 con 0x561219718fb0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3994004030 con 0x561219718fb0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3994011bc0 con 0x561219718fb0 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f39abd96740 10 client.15276 fetch_fsmap learned FSMap version 24 2024-04-24T22:42:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56121979b280 con 0x561219718fb0 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.566+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3994004030 con 0x561219718fb0 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f399403a070 con 0x561219718fb0 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 10 client.15276 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 20 client.15276 populate_metadata read hostname 'smithi111' 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56121979b970 0x56121979dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 4 client.15276 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:42:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561219684340 con 0x56121979b970 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.567+0000 7f39a1400640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56121979b970 0x56121979dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.568+0000 7f3983e00640 10 client.15276 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.568+0000 7f39a1400640 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56121979b970 0x56121979dd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aadd350616162230 server_cookie=f8c91468cf9572ae in_seq=0 out_seq=0 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f3983e00640 1 -- 172.21.15.111:0/4214729152 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561219684340 con 0x56121979b970 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f3983e00640 10 client.15276 handle_command_reply: tid=0 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 client.15276 shutdown 2024-04-24T22:42:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 msgr2=0x56121972c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3994029580 tx=0x7f3994029bc0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 shutdown_connections 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56121979b970 0x56121979dd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561219724190 0x561219729640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5612197198d0 0x561219729100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 --2- 172.21.15.111:0/4214729152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561219718fb0 0x56121972c320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 >> 172.21.15.111:0/4214729152 conn(0x5612197154c0 msgr2=0x56121972ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f3983e00640 0 client.15276 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:42:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 shutdown_connections 2024-04-24T22:42:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 1 -- 172.21.15.111:0/4214729152 wait complete. 2024-04-24T22:42:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.572+0000 7f39abd96740 20 client.15276 trim_cache size 0 max 16384 2024-04-24T22:42:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:42:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:42:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:42:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:42:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 msgr2=0x7f39880a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 0x7f39880a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3994009620 tx=0x7f3994005f90 comp rx=0 tx=0).stop 2024-04-24T22:42:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 msgr2=0x7f39a419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a419a650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9f0 tx=0x7f399000eeb0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 shutdown_connections 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a410de00 0x7f39a419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f39880a2ea0 0x7f39880a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a4074430 0x7f39a419a650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 --2- 172.21.15.111:0/1914224211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a4073b10 0x7f39a419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 >> 172.21.15.111:0/1914224211 conn(0x7f39a40fd090 msgr2=0x7f39a40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 shutdown_connections 2024-04-24T22:42:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:42:52.573+0000 7f39a3200640 1 -- 172.21.15.111:0/1914224211 wait complete. 2024-04-24T22:42:54.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:54 smithi171 ceph-mon[32337]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-24T22:42:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:54 smithi186 ceph-mon[32753]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-24T22:42:54.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:54 smithi111 ceph-mon[27746]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-24T22:42:56.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:56 smithi171 ceph-mon[32337]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T22:42:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:56 smithi186 ceph-mon[32753]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T22:42:56.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:56 smithi111 ceph-mon[27746]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T22:42:58.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:42:58 smithi171 ceph-mon[32337]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:42:58.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:42:58 smithi186 ceph-mon[32753]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:42:58.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:42:58 smithi111 ceph-mon[27746]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:43:00.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:00 smithi171 ceph-mon[32337]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:43:00.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:00 smithi186 ceph-mon[32753]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:43:00.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:00 smithi111 ceph-mon[27746]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T22:43:02.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:02 smithi171 ceph-mon[32337]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:43:02.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:02 smithi186 ceph-mon[32753]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:43:02.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:02 smithi111 ceph-mon[27746]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T22:43:04.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:04 smithi171 ceph-mon[32337]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-24T22:43:04.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:04 smithi186 ceph-mon[32753]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-24T22:43:04.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:04 smithi111 ceph-mon[27746]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 -- 172.21.15.111:0/1911102353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558101560 msgr2=0x7f65581019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 --2- 172.21.15.111:0/1911102353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558101560 0x7f65581019e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a3f0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 -- 172.21.15.111:0/1911102353 shutdown_connections 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 --2- 172.21.15.111:0/1911102353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101f20 0x7f655810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 --2- 172.21.15.111:0/1911102353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558101560 0x7f65581019e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 --2- 172.21.15.111:0/1911102353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558107620 0x7f6558107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 -- 172.21.15.111:0/1911102353 >> 172.21.15.111:0/1911102353 conn(0x7f65580fd090 msgr2=0x7f65580ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 -- 172.21.15.111:0/1911102353 shutdown_connections 2024-04-24T22:43:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.763+0000 7f655d800640 1 -- 172.21.15.111:0/1911102353 wait complete. 2024-04-24T22:43:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 Processor -- start 2024-04-24T22:43:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 -- start start 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558101f20 0x7f655810ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558107620 0x7f655810bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6558075610 con 0x7f6558107620 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6558075490 con 0x7f6558101f20 2024-04-24T22:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.765+0000 7f655d800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6558075790 con 0x7f6558101560 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f655ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558107620 0x7f655810bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53912/0 (socket says 172.21.15.111:53912) 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 -- 172.21.15.111:0/3977507573 learned_addr learned my addr 172.21.15.111:0/3977507573 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557400640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558101f20 0x7f655810ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 -- 172.21.15.111:0/3977507573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558101f20 msgr2=0x7f655810ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558101f20 0x7f655810ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 -- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558107620 msgr2=0x7f655810bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558107620 0x7f655810bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.766+0000 7f6557e00640 1 -- 172.21.15.111:0/3977507573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65581aa670 con 0x7f6558101560 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f6557e00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f654800ea90 tx=0x7f654800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f6554c00640 1 -- 172.21.15.111:0/3977507573 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654800ce40 con 0x7f6558101560 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f6554c00640 1 -- 172.21.15.111:0/3977507573 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65480035f0 con 0x7f6558101560 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65581aa960 con 0x7f6558101560 2024-04-24T22:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f6554c00640 1 -- 172.21.15.111:0/3977507573 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548010690 con 0x7f6558101560 2024-04-24T22:43:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.767+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581aae80 con 0x7f6558101560 2024-04-24T22:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 Processor -- start 2024-04-24T22:43:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 -- start start 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6554c00640 1 -- 172.21.15.111:0/3977507573 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f65480037a0 con 0x7f6558101560 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66005f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660069c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b65ffc2ab0 con 0x55b66005efb0 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b65ff6b640 con 0x55b66005f8d0 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.772+0000 7f6561c42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b65ffbea70 con 0x55b66006a190 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53916/0 (socket says 172.21.15.111:53916) 2024-04-24T22:43:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 -- 172.21.15.111:0/3569168609 learned_addr learned my addr 172.21.15.111:0/3569168609 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f655ce00640 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 -- 172.21.15.111:0/3569168609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 msgr2=0x55b660069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 -- 172.21.15.111:0/3569168609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 msgr2=0x55b66005f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66005f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.773+0000 7f6557400640 1 -- 172.21.15.111:0/3569168609 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65fcd7210 con 0x55b66006a190 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.774+0000 7f6557400640 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f6544004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c525a1733ea2b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.774+0000 7f654d600640 1 -- 172.21.15.111:0/3569168609 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544035040 con 0x55b66006a190 2024-04-24T22:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.774+0000 7f6554c00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 0x7f653c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.775+0000 7f654d600640 1 -- 172.21.15.111:0/3569168609 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6544005520 con 0x55b66006a190 2024-04-24T22:43:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.775+0000 7f654d600640 1 -- 172.21.15.111:0/3569168609 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440035f0 con 0x55b66006a190 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.775+0000 7f6554c00640 1 -- 172.21.15.111:0/3977507573 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f65480cfaa0 con 0x7f6558101560 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.775+0000 7f6561c42740 1 -- 172.21.15.111:0/3569168609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 msgr2=0x55b66006c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f6544004040 comp rx=0 tx=0).stop 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f655ce00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 0x7f653c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 -- 172.21.15.111:0/3569168609 shutdown_connections 2024-04-24T22:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b66006c580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 --2- 172.21.15.111:0/3569168609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66005f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 -- 172.21.15.111:0/3569168609 >> 172.21.15.111:0/3569168609 conn(0x55b66005b4c0 msgr2=0x55b66005b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 -- 172.21.15.111:0/3569168609 shutdown_connections 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.776+0000 7f6561c42740 1 -- 172.21.15.111:0/3569168609 wait complete. 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.777+0000 7f655ce00640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 0x7f653c0a53b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6550005fd0 tx=0x7f6550009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.777+0000 7f6561c42740 1 -- start start 2024-04-24T22:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.777+0000 7f6561c42740 1 Processor -- start 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66005efb0 0x55b660067060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66006a190 0x55b660067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b66006cd60 con 0x55b66005f8d0 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b65ff7e990 con 0x55b66006a190 2024-04-24T22:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b66006cee0 con 0x55b66005efb0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6561c42740 10 client.0 _ll_register_callbacks cb 0x55b65ff80950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6557400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6557400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44482/0 (socket says 172.21.15.111:44482) 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.778+0000 7f6557400640 1 -- 172.21.15.111:0/3708706398 learned_addr learned my addr 172.21.15.111:0/3708706398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557e00640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66005efb0 0x55b660067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f655ce00640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66006a190 0x55b660067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557400640 1 -- 172.21.15.111:0/3708706398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66005efb0 msgr2=0x55b660067060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557400640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66005efb0 0x55b660067060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557400640 1 -- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66006a190 msgr2=0x55b660067ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557400640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66006a190 0x55b660067ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.779+0000 7f6557400640 1 -- 172.21.15.111:0/3708706398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65fda3620 con 0x55b66005f8d0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6557400640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6544005ab0 tx=0x7f6544004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcfe1f44ab9877c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6542000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544051020 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f6544004da0 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6544005520 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6544035040 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.780+0000 7f6561c42740 10 client.15288 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b660078bf0 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6544005520 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f654404c070 con 0x55b66005f8d0 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 10 client.15288 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 10 client.15288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 20 client.15288 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b660079f40 0x55b66007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 4 client.15288 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:43:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.781+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b65ffcfff0 con 0x55b660079f40 2024-04-24T22:43:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.782+0000 7f6557e00640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b660079f40 0x55b66007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.783+0000 7f6557e00640 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b660079f40 0x55b66007c330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37e995b8b14dbd8 server_cookie=8325de69c9c49910 in_seq=0 out_seq=0 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.783+0000 7f6542000640 10 client.15288 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6542000640 1 -- 172.21.15.111:0/3708706398 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b65ffcfff0 con 0x55b660079f40 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6542000640 10 client.15288 handle_command_reply: tid=0 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 client.15288 shutdown 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 msgr2=0x55b6600675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6544005ab0 tx=0x7f6544004520 comp rx=0 tx=0).stop 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 shutdown_connections 2024-04-24T22:43:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b660079f40 0x55b66007c330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66006a190 0x55b660067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005f8d0 0x55b6600675a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 --2- 172.21.15.111:0/3708706398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66005efb0 0x55b660067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6542000640 0 client.15288 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.784+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 >> 172.21.15.111:0/3708706398 conn(0x55b66005b4c0 msgr2=0x55b660061240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.785+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 shutdown_connections 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.785+0000 7f6561c42740 1 -- 172.21.15.111:0/3708706398 wait complete. 2024-04-24T22:43:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.785+0000 7f6561c42740 20 client.15288 trim_cache size 0 max 16384 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.815+0000 7f6561c42740 1 Processor -- start 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.815+0000 7f6561c42740 1 -- start start 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.815+0000 7f6561c42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6561c42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6561c42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b660064320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6561c42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b60e0 con 0x55b66005efb0 2024-04-24T22:43:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6561c42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b39b0 con 0x55b66005f8d0 2024-04-24T22:43:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6561c42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b1800 con 0x55b66006a190 2024-04-24T22:43:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44490/0 (socket says 172.21.15.111:44490) 2024-04-24T22:43:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 -- 172.21.15.111:0/1660346428 learned_addr learned my addr 172.21.15.111:0/1660346428 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 -- 172.21.15.111:0/1660346428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 msgr2=0x55b660064320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:43:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b660064320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.816+0000 7f6557e00640 1 -- 172.21.15.111:0/1660346428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 msgr2=0x55b660068fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f6557e00640 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660068fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f6557e00640 1 -- 172.21.15.111:0/1660346428 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b660061eb0 con 0x55b66005efb0 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f6557e00640 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6548098780 tx=0x7f6548003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778c988800b3722f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f654cc00640 1 -- 172.21.15.111:0/1660346428 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654809bdb0 con 0x55b66005efb0 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f654cc00640 1 -- 172.21.15.111:0/1660346428 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65480a0160 con 0x55b66005efb0 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.817+0000 7f654cc00640 1 -- 172.21.15.111:0/1660346428 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548003fd0 con 0x55b66005efb0 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 -- 172.21.15.111:0/1660346428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 msgr2=0x55b660061970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6548098780 tx=0x7f6548003c90 comp rx=0 tx=0).stop 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 -- 172.21.15.111:0/1660346428 shutdown_connections 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b660064320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b660068fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 --2- 172.21.15.111:0/1660346428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b660061970 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.818+0000 7f6561c42740 1 -- 172.21.15.111:0/1660346428 >> 172.21.15.111:0/1660346428 conn(0x55b66005b4c0 msgr2=0x55b660068890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.819+0000 7f6561c42740 1 -- 172.21.15.111:0/1660346428 shutdown_connections 2024-04-24T22:43:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.819+0000 7f6561c42740 1 -- 172.21.15.111:0/1660346428 wait complete. 2024-04-24T22:43:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.819+0000 7f6561c42740 1 -- start start 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.819+0000 7f6561c42740 1 Processor -- start 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66007d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b6600d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b1e70 con 0x55b66005efb0 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b32e0 con 0x55b66005f8d0 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b2540 con 0x55b66006a190 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6561c42740 10 client.0 _ll_register_callbacks cb 0x7f6544009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.820+0000 7f6557e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66007d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34514/0 (socket says 172.21.15.111:34514) 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 -- 172.21.15.111:0/4001518968 learned_addr learned my addr 172.21.15.111:0/4001518968 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f655ce00640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b6600d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 -- 172.21.15.111:0/4001518968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 msgr2=0x55b6600d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b6600d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 -- 172.21.15.111:0/4001518968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 msgr2=0x55b66007d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66007d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 -- 172.21.15.111:0/4001518968 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6600d96e0 con 0x55b66005f8d0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557e00640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66007d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6557400640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6544003e90 tx=0x7f6544032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd3f427b93517b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.821+0000 7f6542000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544051030 con 0x55b66005f8d0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f6544003a00 con 0x55b66005f8d0 2024-04-24T22:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65480a0160 con 0x55b66005f8d0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6544003c20 con 0x55b66005f8d0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6561c42740 10 client.25376 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.822+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b6600e0a70 con 0x55b66005f8d0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65480a0160 con 0x55b66005f8d0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f654404c040 con 0x55b66005f8d0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 10 client.25376 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 20 client.25376 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b6600e1100 0x55b6600e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 4 client.25376 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b65ffca340 con 0x55b6600e1100 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.823+0000 7f6557e00640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b6600e1100 0x55b6600e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.824+0000 7f6557e00640 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b6600e1100 0x55b6600e3520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3294578687e89e4 server_cookie=9f8e5e6ff97d0b95 in_seq=0 out_seq=0 2024-04-24T22:43:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.824+0000 7f6542000640 10 client.25376 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6542000640 1 -- 172.21.15.111:0/4001518968 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b65ffca340 con 0x55b6600e1100 2024-04-24T22:43:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6542000640 10 client.25376 handle_command_reply: tid=0 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 client.25376 shutdown 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 msgr2=0x55b66007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6544003e90 tx=0x7f6544032c80 comp rx=0 tx=0).stop 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 shutdown_connections 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b6600e1100 0x55b6600e3520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.831+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66006a190 0x55b6600d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b66005f8d0 0x55b66007db20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 1 --2- 172.21.15.111:0/4001518968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b66005efb0 0x55b66007d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 >> 172.21.15.111:0/4001518968 conn(0x55b66005b4c0 msgr2=0x55b660069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6542000640 0 client.25376 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 shutdown_connections 2024-04-24T22:43:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 1 -- 172.21.15.111:0/4001518968 wait complete. 2024-04-24T22:43:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.832+0000 7f6561c42740 20 client.25376 trim_cache size 0 max 16384 2024-04-24T22:43:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:43:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:43:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:43:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 msgr2=0x7f653c0a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 0x7f653c0a53b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6550005fd0 tx=0x7f6550009500 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 msgr2=0x7f655810b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f654800ea90 tx=0x7f654800ef50 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 shutdown_connections 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6558107620 0x7f655810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6558101f20 0x7f655810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f653c0a2ef0 0x7f653c0a53b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 --2- 172.21.15.111:0/3977507573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6558101560 0x7f655810b550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 >> 172.21.15.111:0/3977507573 conn(0x7f65580fd090 msgr2=0x7f65580fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 shutdown_connections 2024-04-24T22:43:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:05.833+0000 7f655d800640 1 -- 172.21.15.111:0/3977507573 wait complete. 2024-04-24T22:43:06.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:06 smithi171 ceph-mon[32337]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:06.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:06 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:06.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:06 smithi186 ceph-mon[32753]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:06.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:06 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:06.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:06 smithi111 ceph-mon[27746]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:06 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:43:07.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:43:07.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:07 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:08.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:08.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:08.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:43:08.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:43:08.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:07 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:08.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T22:43:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:43:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:43:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:07 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:43:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:09 smithi171 ceph-mon[32337]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T22:43:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:09 smithi186 ceph-mon[32753]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T22:43:10.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:09 smithi111 ceph-mon[27746]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T22:43:11.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:11 smithi171 ceph-mon[32337]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T22:43:11.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:11 smithi186 ceph-mon[32753]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T22:43:12.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:11 smithi111 ceph-mon[27746]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T22:43:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:13 smithi171 ceph-mon[32337]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T22:43:14.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:13 smithi186 ceph-mon[32753]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T22:43:14.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:13 smithi111 ceph-mon[27746]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T22:43:15.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:15 smithi171 ceph-mon[32337]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-24T22:43:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:15 smithi186 ceph-mon[32753]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-24T22:43:16.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:15 smithi111 ceph-mon[27746]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-24T22:43:17.452 INFO:tasks.workunit.client.0.smithi111.stderr:Updating files: 63% (7911/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T22:43:17.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:17 smithi171 ceph-mon[32337]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T22:43:17.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:17 smithi186 ceph-mon[32753]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T22:43:18.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:17 smithi111 ceph-mon[27746]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T22:43:18.073 INFO:tasks.workunit.client.0.smithi111.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T22:43:18.073 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.073 INFO:tasks.workunit.client.0.smithi111.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T22:43:18.073 INFO:tasks.workunit.client.0.smithi111.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T22:43:18.073 INFO:tasks.workunit.client.0.smithi111.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: git switch -c 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr:Or undo this operation with: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: git switch - 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T22:43:18.074 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-04-24T22:43:18.075 INFO:tasks.workunit.client.0.smithi111.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T22:43:18.082 DEBUG:teuthology.orchestra.run.smithi111:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T22:43:18.195 INFO:tasks.workunit.client.0.smithi111.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T22:43:18.202 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:43:18.202 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.972+0000 7f280be00640 1 -- 172.21.15.111:0/3579309181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c073b70 msgr2=0x7f280c073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.972+0000 7f280be00640 1 --2- 172.21.15.111:0/3579309181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c073b70 0x7f280c073f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009900 tx=0x7f27fc02a480 comp rx=0 tx=0).stop 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 -- 172.21.15.111:0/3579309181 shutdown_connections 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 --2- 172.21.15.111:0/3579309181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 0x7f280c1101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 --2- 172.21.15.111:0/3579309181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c074490 0x7f280c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 --2- 172.21.15.111:0/3579309181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c073b70 0x7f280c073f50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 -- 172.21.15.111:0/3579309181 >> 172.21.15.111:0/3579309181 conn(0x7f280c0fd090 msgr2=0x7f280c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 -- 172.21.15.111:0/3579309181 shutdown_connections 2024-04-24T22:43:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.973+0000 7f280be00640 1 -- 172.21.15.111:0/3579309181 wait complete. 2024-04-24T22:43:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 Processor -- start 2024-04-24T22:43:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 -- start start 2024-04-24T22:43:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c073b70 0x7f280c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 0x7f280c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f280c113b30 con 0x7f280c074490 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1139b0 con 0x7f280c10dd80 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f280c113cb0 con 0x7f280c073b70 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51068/0 (socket says 172.21.15.111:51068) 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280a000640 1 -- 172.21.15.111:0/931868022 learned_addr learned my addr 172.21.15.111:0/931868022 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280b400640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 0x7f280c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.974+0000 7f280aa00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c073b70 0x7f280c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 -- 172.21.15.111:0/931868022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c073b70 msgr2=0x7f280c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c073b70 0x7f280c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 -- 172.21.15.111:0/931868022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 msgr2=0x7f280c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 0x7f280c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 -- 172.21.15.111:0/931868022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c19f000 con 0x7f280c074490 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280a000640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9f0 tx=0x7f27f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f2803400640 1 -- 172.21.15.111:0/931868022 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800cd80 con 0x7f280c074490 2024-04-24T22:43:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f2803400640 1 -- 172.21.15.111:0/931868022 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f8010910 con 0x7f280c074490 2024-04-24T22:43:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f2803400640 1 -- 172.21.15.111:0/931868022 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8018990 con 0x7f280c074490 2024-04-24T22:43:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.975+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c19f2f0 con 0x7f280c074490 2024-04-24T22:43:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.976+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1a6b80 con 0x7f280c074490 2024-04-24T22:43:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 Processor -- start 2024-04-24T22:43:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 -- start start 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc398d0 0x5555ebc43c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555ebb9cab0 con 0x5555ebc44190 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555ebb45640 con 0x5555ebc398d0 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.980+0000 7f281464d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5555ebb98a70 con 0x5555ebc38fb0 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f2803400640 1 -- 172.21.15.111:0/931868022 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f27f8010ac0 con 0x7f280c074490 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52916/0 (socket says 172.21.15.111:52916) 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 -- 172.21.15.111:0/462308888 learned_addr learned my addr 172.21.15.111:0/462308888 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280aa00640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc398d0 0x5555ebc43c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280a000640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 -- 172.21.15.111:0/462308888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc398d0 msgr2=0x5555ebc43c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc398d0 0x5555ebc43c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 -- 172.21.15.111:0/462308888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 msgr2=0x5555ebc46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc46580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280b400640 1 -- 172.21.15.111:0/462308888 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eb8b1210 con 0x5555ebc38fb0 2024-04-24T22:43:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.981+0000 7f280a000640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:43:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f280b400640 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2804009f50 tx=0x7f28040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41e24f97da52291 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f27f7e00640 1 -- 172.21.15.111:0/462308888 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804004d20 con 0x5555ebc38fb0 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f27f7e00640 1 -- 172.21.15.111:0/462308888 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804004050 con 0x5555ebc38fb0 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f27f7e00640 1 -- 172.21.15.111:0/462308888 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804005280 con 0x5555ebc38fb0 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f2803400640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 0x7f27f00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.982+0000 7f2803400640 1 -- 172.21.15.111:0/931868022 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f27f80cf9f0 con 0x7f280c074490 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f280aa00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 0x7f27f00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 -- 172.21.15.111:0/462308888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 msgr2=0x5555ebc39390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2804009f50 tx=0x7f28040118d0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 -- 172.21.15.111:0/462308888 shutdown_connections 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc398d0 0x5555ebc43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 --2- 172.21.15.111:0/462308888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc39390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 -- 172.21.15.111:0/462308888 >> 172.21.15.111:0/462308888 conn(0x5555ebc354c0 msgr2=0x5555ebc358a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 -- 172.21.15.111:0/462308888 shutdown_connections 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f281464d740 1 -- 172.21.15.111:0/462308888 wait complete. 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.983+0000 7f280aa00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 0x7f27f00a5360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc005b50 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 -- start start 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 Processor -- start 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555ebc46dc0 con 0x5555ebc398d0 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555ebb58990 con 0x5555ebc44190 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5555ebc46f40 con 0x5555ebc38fb0 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.984+0000 7f281464d740 10 client.0 _ll_register_callbacks cb 0x5555ebb5a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51072/0 (socket says 172.21.15.111:51072) 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 -- 172.21.15.111:0/3682535688 learned_addr learned my addr 172.21.15.111:0/3682535688 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280aa00640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280b400640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 -- 172.21.15.111:0/3682535688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 msgr2=0x5555ebc41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc41070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 -- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 msgr2=0x5555ebc41af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc41af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 -- 172.21.15.111:0/3682535688 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eb97d620 con 0x5555ebc398d0 2024-04-24T22:43:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f280a000640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9c0 tx=0x7f27f8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8b3ce7febfa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f27f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.985+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f80a4970 con 0x5555ebc398d0 2024-04-24T22:43:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f27f809b200 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2804004050 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f27f80b9050 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 10 client.15303 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5555ebc52b40 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804004050 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f27f8086040 con 0x5555ebc398d0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 10 client.15303 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 20 client.15303 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebc53fb0 0x5555ebc563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 4 client.15303 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.986+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ebba9ff0 con 0x5555ebc53fb0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.987+0000 7f280aa00640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebc53fb0 0x5555ebc563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.987+0000 7f280aa00640 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebc53fb0 0x5555ebc563a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6badc3550b30ac44 server_cookie=62eda291b2e32047 in_seq=0 out_seq=0 2024-04-24T22:43:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.987+0000 7f27f4c00640 10 client.15303 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f27f4c00640 1 -- 172.21.15.111:0/3682535688 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555ebba9ff0 con 0x5555ebc53fb0 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f27f4c00640 10 client.15303 handle_command_reply: tid=0 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 client.15303 shutdown 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 msgr2=0x5555ebc415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9c0 tx=0x7f27f8011e20 comp rx=0 tx=0).stop 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 shutdown_connections 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebc53fb0 0x5555ebc563a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc41af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc398d0 0x5555ebc415b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 --2- 172.21.15.111:0/3682535688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 >> 172.21.15.111:0/3682535688 conn(0x5555ebc354c0 msgr2=0x5555ebc3b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f27f4c00640 0 client.15303 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 shutdown_connections 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.988+0000 7f281464d740 1 -- 172.21.15.111:0/3682535688 wait complete. 2024-04-24T22:43:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:18.989+0000 7f281464d740 20 client.15303 trim_cache size 0 max 16384 2024-04-24T22:43:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.004+0000 7f281464d740 1 Processor -- start 2024-04-24T22:43:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.004+0000 7f281464d740 1 -- start start 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc438b0 0x5555ebc5bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1b1c90 con 0x5555ebc44190 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1af560 con 0x5555ebc438b0 2024-04-24T22:43:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f281464d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ad3b0 con 0x5555ebc38fb0 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51076/0 (socket says 172.21.15.111:51076) 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 -- 172.21.15.111:0/3777708053 learned_addr learned my addr 172.21.15.111:0/3777708053 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280aa00640 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280b400640 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc438b0 0x5555ebc5bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 -- 172.21.15.111:0/3777708053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 msgr2=0x5555ebc34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 -- 172.21.15.111:0/3777708053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc438b0 msgr2=0x5555ebc5bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.005+0000 7f280a000640 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc438b0 0x5555ebc5bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.006+0000 7f280a000640 1 -- 172.21.15.111:0/3777708053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ebc352e0 con 0x5555ebc44190 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.006+0000 7f280a000640 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27f8098210 tx=0x7f27f809b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98baefff4edb8690 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.006+0000 7f27f7400640 1 -- 172.21.15.111:0/3777708053 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8098050 con 0x5555ebc44190 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.006+0000 7f27f7400640 1 -- 172.21.15.111:0/3777708053 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f80a4dc0 con 0x5555ebc44190 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.006+0000 7f27f7400640 1 -- 172.21.15.111:0/3777708053 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f80cd240 con 0x5555ebc44190 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- 172.21.15.111:0/3777708053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 msgr2=0x5555ebc3b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27f8098210 tx=0x7f27f809b9c0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- 172.21.15.111:0/3777708053 shutdown_connections 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc438b0 0x5555ebc5bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc44190 0x5555ebc3b970 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 --2- 172.21.15.111:0/3777708053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- 172.21.15.111:0/3777708053 >> 172.21.15.111:0/3777708053 conn(0x5555ebc354c0 msgr2=0x5555ebc418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- 172.21.15.111:0/3777708053 shutdown_connections 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- 172.21.15.111:0/3777708053 wait complete. 2024-04-24T22:43:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.007+0000 7f281464d740 1 -- start start 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 Processor -- start 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ada20 con 0x5555ebc438b0 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1aee90 con 0x5555ebc44190 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ae0f0 con 0x5555ebc38fb0 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f281464d740 10 client.0 _ll_register_callbacks cb 0x7f27f800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f280aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f280a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51090/0 (socket says 172.21.15.111:51090) 2024-04-24T22:43:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.008+0000 7f280a000640 1 -- 172.21.15.111:0/454744331 learned_addr learned my addr 172.21.15.111:0/454744331 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280b400640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 -- 172.21.15.111:0/454744331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 msgr2=0x5555ebc4c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc4c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 -- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 msgr2=0x5555ebc496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 -- 172.21.15.111:0/454744331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ebc49c10 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f280a000640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27f80cd070 tx=0x7f27f809b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78482196ae0bbe46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f27f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8098050 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f27f8010430 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.009+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f80a4dc0 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f27f8010670 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 10 client.15309 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5555ebcbb2a0 con 0x5555ebc438b0 2024-04-24T22:43:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f80a4dc0 con 0x5555ebc438b0 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f27f8086070 con 0x5555ebc438b0 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 10 client.15309 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 20 client.15309 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebcbb5e0 0x5555ebcbd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 4 client.15309 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555ebba4340 con 0x5555ebcbb5e0 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.010+0000 7f280aa00640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebcbb5e0 0x5555ebcbd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.011+0000 7f280aa00640 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebcbb5e0 0x5555ebcbd9d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7876f509b317e2 server_cookie=c7366eb27064b3ee in_seq=0 out_seq=0 2024-04-24T22:43:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.011+0000 7f27f4c00640 10 client.15309 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f27f4c00640 1 -- 172.21.15.111:0/454744331 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555ebba4340 con 0x5555ebcbb5e0 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f27f4c00640 10 client.15309 handle_command_reply: tid=0 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 client.15309 shutdown 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 msgr2=0x5555ebc49160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27f80cd070 tx=0x7f27f809b640 comp rx=0 tx=0).stop 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 shutdown_connections 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5555ebcbb5e0 0x5555ebcbd9d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555ebc44190 0x5555ebc496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555ebc438b0 0x5555ebc49160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 --2- 172.21.15.111:0/454744331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555ebc38fb0 0x5555ebc4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f27f4c00640 0 client.15309 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 >> 172.21.15.111:0/454744331 conn(0x5555ebc354c0 msgr2=0x5555ebc428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 shutdown_connections 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 1 -- 172.21.15.111:0/454744331 wait complete. 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.017+0000 7f281464d740 20 client.15309 trim_cache size 0 max 16384 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:43:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:43:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 msgr2=0x7f27f00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 0x7f27f00a5360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc005b50 comp rx=0 tx=0).stop 2024-04-24T22:43:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 msgr2=0x7f280c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9f0 tx=0x7f27f800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 shutdown_connections 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f280c10dd80 0x7f280c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f280c074490 0x7f280c19a550 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f27f00a2ea0 0x7f27f00a5360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 --2- 172.21.15.111:0/931868022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c073b70 0x7f280c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 >> 172.21.15.111:0/931868022 conn(0x7f280c0fd090 msgr2=0x7f280c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.018+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 shutdown_connections 2024-04-24T22:43:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:19.019+0000 7f280be00640 1 -- 172.21.15.111:0/931868022 wait complete. 2024-04-24T22:43:19.022 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T22:43:19.120 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T22:43:19.202 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:43:19.208 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:43:19.208 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T22:43:19.289 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:43:19.298 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:43:19.298 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T22:43:19.357 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-24T22:43:19.359 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-24T22:43:19.359 DEBUG:teuthology.orchestra.run.smithi111:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-24T22:43:19.476 INFO:tasks.workunit.client.0.smithi111.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-24T22:43:19.478 INFO:tasks.workunit.client.0.smithi111.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-24T22:43:19.478 INFO:tasks.workunit.client.0.smithi111.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-24T22:43:19.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:19 smithi171 ceph-mon[32337]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T22:43:20.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:19 smithi186 ceph-mon[32753]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T22:43:20.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:19 smithi111 ceph-mon[27746]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T22:43:21.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:21 smithi171 ceph-mon[32337]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:43:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:21 smithi186 ceph-mon[32753]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:43:22.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:21 smithi111 ceph-mon[27746]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:43:23.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:23 smithi171 ceph-mon[32337]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-24T22:43:23.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:23 smithi186 ceph-mon[32753]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-24T22:43:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:23 smithi111 ceph-mon[27746]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-24T22:43:25.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:25 smithi171 ceph-mon[32337]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:26.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:25 smithi186 ceph-mon[32753]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:25 smithi111 ceph-mon[27746]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:27.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:27 smithi171 ceph-mon[32337]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:27.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:27 smithi186 ceph-mon[32753]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:28.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:27 smithi111 ceph-mon[27746]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-24T22:43:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:29 smithi171 ceph-mon[32337]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-24T22:43:29.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:29 smithi171 ceph-mon[32337]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-24T22:43:29.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:29 smithi186 ceph-mon[32753]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-24T22:43:30.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:29 smithi186 ceph-mon[32753]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-24T22:43:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:29 smithi111 ceph-mon[27746]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-24T22:43:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:29 smithi111 ceph-mon[27746]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-24T22:43:31.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:31 smithi171 ceph-mon[32337]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-24T22:43:31.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:31 smithi186 ceph-mon[32753]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-24T22:43:32.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:31 smithi111 ceph-mon[27746]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-24T22:43:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 -- 172.21.15.111:0/1473531758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 msgr2=0x7ff238073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 --2- 172.21.15.111:0/1473531758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff238073df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff228009900 tx=0x7ff22802a480 comp rx=0 tx=0).stop 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 -- 172.21.15.111:0/1473531758 shutdown_connections 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 --2- 172.21.15.111:0/1473531758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff23810e030 0x7ff238110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 --2- 172.21.15.111:0/1473531758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff238074330 0x7ff23810daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 --2- 172.21.15.111:0/1473531758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff238073df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 -- 172.21.15.111:0/1473531758 >> 172.21.15.111:0/1473531758 conn(0x7ff238078bd0 msgr2=0x7ff238078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.159+0000 7ff23fc00640 1 -- 172.21.15.111:0/1473531758 shutdown_connections 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.160+0000 7ff23fc00640 1 -- 172.21.15.111:0/1473531758 wait complete. 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.160+0000 7ff23fc00640 1 Processor -- start 2024-04-24T22:43:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 -- start start 2024-04-24T22:43:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238074330 0x7ff23819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff238113d30 con 0x7ff23810e030 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff238113bb0 con 0x7ff238073a10 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23fc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff238113eb0 con 0x7ff238074330 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53738/0 (socket says 172.21.15.111:53738) 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23dc00640 1 -- 172.21.15.111:0/3224205681 learned_addr learned my addr 172.21.15.111:0/3224205681 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff23d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43120/0 (socket says 172.21.15.111:43120) 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.161+0000 7ff22fe00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238074330 0x7ff23819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 -- 172.21.15.111:0/3224205681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238074330 msgr2=0x7ff23819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238074330 0x7ff23819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 -- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 msgr2=0x7ff23819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 -- 172.21.15.111:0/3224205681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23819f1b0 con 0x7ff238073a10 2024-04-24T22:43:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23dc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff23d200640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff228005940 tx=0x7ff2280040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.162+0000 7ff236000640 1 -- 172.21.15.111:0/3224205681 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22803f030 con 0x7ff238073a10 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.163+0000 7ff236000640 1 -- 172.21.15.111:0/3224205681 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2280044c0 con 0x7ff238073a10 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.163+0000 7ff236000640 1 -- 172.21.15.111:0/3224205681 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228035070 con 0x7ff238073a10 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.163+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23819f440 con 0x7ff238073a10 2024-04-24T22:43:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.163+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a6d30 con 0x7ff238073a10 2024-04-24T22:43:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 Processor -- start 2024-04-24T22:43:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 -- start start 2024-04-24T22:43:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff236000640 1 -- 172.21.15.111:0/3224205681 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff228042030 con 0x7ff238073a10 2024-04-24T22:43:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b8573aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5624b84ccab0 con 0x5624b8568e00 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624b8475640 con 0x5624b8573fe0 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.167+0000 7ff242a13740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5624b84c8a70 con 0x5624b8569720 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53742/0 (socket says 172.21.15.111:53742) 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 -- 172.21.15.111:0/42668832 learned_addr learned my addr 172.21.15.111:0/42668832 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23d200640 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b8573aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff22fe00640 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 -- 172.21.15.111:0/42668832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 msgr2=0x5624b8573aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b8573aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 -- 172.21.15.111:0/42668832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 msgr2=0x5624b85763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85763d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.168+0000 7ff23dc00640 1 -- 172.21.15.111:0/42668832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b81e1210 con 0x5624b8568e00 2024-04-24T22:43:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff23dc00640 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff230011820 tx=0x7ff230011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a173c968ee5d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff22e000640 1 -- 172.21.15.111:0/42668832 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230011330 con 0x5624b8568e00 2024-04-24T22:43:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff22e000640 1 -- 172.21.15.111:0/42668832 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23000fe80 con 0x5624b8568e00 2024-04-24T22:43:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff236000640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 0x7ff21c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff22e000640 1 -- 172.21.15.111:0/42668832 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2300193e0 con 0x5624b8568e00 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff236000640 1 -- 172.21.15.111:0/3224205681 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff2280ef610 con 0x7ff238073a10 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.169+0000 7ff23d200640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 0x7ff21c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- 172.21.15.111:0/42668832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 msgr2=0x5624b85691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff230011820 tx=0x7ff230011ce0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- 172.21.15.111:0/42668832 shutdown_connections 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b8573aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 --2- 172.21.15.111:0/42668832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b85691e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- 172.21.15.111:0/42668832 >> 172.21.15.111:0/42668832 conn(0x5624b8565310 msgr2=0x5624b85656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- 172.21.15.111:0/42668832 shutdown_connections 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- 172.21.15.111:0/42668832 wait complete. 2024-04-24T22:43:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff23d200640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 0x7ff21c0a52e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff2280ece40 tx=0x7ff2280f5550 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.170+0000 7ff242a13740 1 -- start start 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 Processor -- start 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b8570f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b85714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5624b8576c10 con 0x5624b8568e00 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624b8488990 con 0x5624b8573fe0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5624b8576d90 con 0x5624b8569720 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff242a13740 10 client.0 _ll_register_callbacks cb 0x5624b848a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff23dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b8570f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.171+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b85714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b85714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:39922/0 (socket says 172.21.15.111:39922) 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff22fe00640 1 -- 172.21.15.111:0/1563747252 learned_addr learned my addr 172.21.15.111:0/1563747252 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 -- 172.21.15.111:0/1563747252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 msgr2=0x5624b85714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b85714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 -- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 msgr2=0x5624b85719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 -- 172.21.15.111:0/1563747252 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b82ad620 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b8570f60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff230009960 tx=0x7ff230010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d909a77eb2175 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff226a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23002d060 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff230004c50 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.172+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5624b84d9ff0 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff230004e50 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 10 client.15321 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624b8582ac0 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5624b84d9ff0 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff230018070 con 0x5624b8568e00 2024-04-24T22:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 20 client.15321 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b8583ec0 0x5624b85862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 4 client.15321 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.173+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b84d4340 con 0x5624b8583ec0 2024-04-24T22:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.174+0000 7ff22fe00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b8583ec0 0x5624b85862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.174+0000 7ff22fe00640 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b8583ec0 0x5624b85862b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c814e61287e54a4 server_cookie=ba77ba592afbbf2 in_seq=0 out_seq=0 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.174+0000 7ff226a00640 10 client.15321 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.175+0000 7ff226a00640 1 -- 172.21.15.111:0/1563747252 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624b84d4340 con 0x5624b8583ec0 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.175+0000 7ff226a00640 10 client.15321 handle_command_reply: tid=0 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.175+0000 7ff242a13740 1 client.15321 shutdown 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 msgr2=0x5624b8570f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b8570f60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff230009960 tx=0x7ff230010b90 comp rx=0 tx=0).stop 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 shutdown_connections 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b8583ec0 0x5624b85862b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8569720 0x5624b85714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 --2- 172.21.15.111:0/1563747252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8568e00 0x5624b8570f60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 >> 172.21.15.111:0/1563747252 conn(0x5624b8565310 msgr2=0x5624b856b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff226a00640 0 client.15321 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 shutdown_connections 2024-04-24T22:43:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 1 -- 172.21.15.111:0/1563747252 wait complete. 2024-04-24T22:43:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.176+0000 7ff242a13740 20 client.15321 trim_cache size 0 max 16384 2024-04-24T22:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.192+0000 7ff242a13740 1 Processor -- start 2024-04-24T22:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.192+0000 7ff242a13740 1 -- start start 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b857fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b858b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381b1e80 con 0x5624b8569720 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381af750 con 0x5624b8573fe0 2024-04-24T22:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff242a13740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381ad5a0 con 0x5624b8568e00 2024-04-24T22:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53754/0 (socket says 172.21.15.111:53754) 2024-04-24T22:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff22fe00640 1 -- 172.21.15.111:0/1910875119 learned_addr learned my addr 172.21.15.111:0/1910875119 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b857fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.193+0000 7ff23dc00640 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b858b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 -- 172.21.15.111:0/1910875119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 msgr2=0x5624b857fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b857fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 -- 172.21.15.111:0/1910875119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 msgr2=0x5624b858b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b858b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 -- 172.21.15.111:0/1910875119 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b8569e90 con 0x5624b8569720 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22fe00640 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5624b8586910 tx=0x7ff220005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e006f4eb515ac4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22d600640 1 -- 172.21.15.111:0/1910875119 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220026070 con 0x5624b8569720 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22d600640 1 -- 172.21.15.111:0/1910875119 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff220004310 con 0x5624b8569720 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.194+0000 7ff22d600640 1 -- 172.21.15.111:0/1910875119 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220003b30 con 0x5624b8569720 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 -- 172.21.15.111:0/1910875119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 msgr2=0x5624b8571800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5624b8586910 tx=0x7ff220005f90 comp rx=0 tx=0).stop 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 -- 172.21.15.111:0/1910875119 shutdown_connections 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b858b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b8571800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 --2- 172.21.15.111:0/1910875119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b857fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 -- 172.21.15.111:0/1910875119 >> 172.21.15.111:0/1910875119 conn(0x5624b8565310 msgr2=0x5624b8564480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 -- 172.21.15.111:0/1910875119 shutdown_connections 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.195+0000 7ff242a13740 1 -- 172.21.15.111:0/1910875119 wait complete. 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 -- start start 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 Processor -- start 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b8577c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381adc10 con 0x5624b8569720 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381af080 con 0x5624b8573fe0 2024-04-24T22:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381ae2e0 con 0x5624b8568e00 2024-04-24T22:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff242a13740 10 client.0 _ll_register_callbacks cb 0x7ff2200097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.196+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53770/0 (socket says 172.21.15.111:53770) 2024-04-24T22:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 -- 172.21.15.111:0/833208522 learned_addr learned my addr 172.21.15.111:0/833208522 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff23dc00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b8577c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff23dc00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 -- 172.21.15.111:0/833208522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 msgr2=0x5624b8577c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b8577c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 -- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 msgr2=0x5624b85786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 -- 172.21.15.111:0/833208522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b85f1cc0 con 0x5624b8569720 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff22fe00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff220003670 tx=0x7ff2200036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb859e89960764 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff226a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.197+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220026040 con 0x5624b8569720 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff220003870 con 0x5624b8569720 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff220004310 con 0x5624b8569720 2024-04-24T22:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff220003a70 con 0x5624b8569720 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624b85f2fb0 con 0x5624b8569720 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff220004310 con 0x5624b8569720 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff22002a4d0 con 0x5624b8569720 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 20 client.15327 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b85f3670 0x5624b85f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.198+0000 7ff242a13740 4 client.15327 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.199+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624b84527a0 con 0x5624b85f3670 2024-04-24T22:43:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.199+0000 7ff23dc00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b85f3670 0x5624b85f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.199+0000 7ff23dc00640 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b85f3670 0x5624b85f5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b75148d195d19b1 server_cookie=d483ad9b4fea2bed in_seq=0 out_seq=0 2024-04-24T22:43:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.199+0000 7ff226a00640 10 client.15327 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.212+0000 7ff226a00640 1 -- 172.21.15.111:0/833208522 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624b84527a0 con 0x5624b85f3670 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.212+0000 7ff226a00640 10 client.15327 handle_command_reply: tid=0 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.212+0000 7ff242a13740 1 client.15327 shutdown 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 msgr2=0x5624b85781b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff220003670 tx=0x7ff2200036a0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 shutdown_connections 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5624b85f3670 0x5624b85f5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624b8573fe0 0x5624b85786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624b8569720 0x5624b85781b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 --2- 172.21.15.111:0/833208522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624b8568e00 0x5624b8577c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff226a00640 0 client.15327 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 >> 172.21.15.111:0/833208522 conn(0x5624b8565310 msgr2=0x5624b856a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 shutdown_connections 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 1 -- 172.21.15.111:0/833208522 wait complete. 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.213+0000 7ff242a13740 20 client.15327 trim_cache size 0 max 16384 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:43:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:43:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 msgr2=0x7ff21c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 0x7ff21c0a52e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff2280ece40 tx=0x7ff2280f5550 comp rx=0 tx=0).stop 2024-04-24T22:43:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 msgr2=0x7ff23819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff228005940 tx=0x7ff2280040c0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 shutdown_connections 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23810e030 0x7ff23819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff21c0a2e20 0x7ff21c0a52e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238074330 0x7ff23819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 --2- 172.21.15.111:0/3224205681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238073a10 0x7ff23819a210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 >> 172.21.15.111:0/3224205681 conn(0x7ff238078bd0 msgr2=0x7ff2380fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 shutdown_connections 2024-04-24T22:43:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:32.214+0000 7ff23fc00640 1 -- 172.21.15.111:0/3224205681 wait complete. 2024-04-24T22:43:33.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:33 smithi171 ceph-mon[32337]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-24T22:43:33.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:33 smithi186 ceph-mon[32753]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-24T22:43:34.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:33 smithi111 ceph-mon[27746]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-24T22:43:35.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:35 smithi171 ceph-mon[32337]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-24T22:43:35.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:35 smithi186 ceph-mon[32753]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-24T22:43:36.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:35 smithi111 ceph-mon[27746]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-24T22:43:37.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:37 smithi171 ceph-mon[32337]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-24T22:43:37.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:37 smithi186 ceph-mon[32753]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-24T22:43:38.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:37 smithi111 ceph-mon[27746]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-24T22:43:39.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:39 smithi171 ceph-mon[32337]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-24T22:43:39.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:39 smithi186 ceph-mon[32753]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-24T22:43:40.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:39 smithi111 ceph-mon[27746]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-24T22:43:41.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:41 smithi186 ceph-mon[32753]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-24T22:43:42.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:41 smithi111 ceph-mon[27746]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-24T22:43:42.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:41 smithi171 ceph-mon[32337]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-24T22:43:43.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:43 smithi186 ceph-mon[32753]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:44.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:43 smithi111 ceph-mon[27746]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:44.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:43 smithi171 ceph-mon[32337]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.354+0000 7fcdc7000640 1 -- 172.21.15.111:0/341120822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc0107fa0 msgr2=0x7fcdc0108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.354+0000 7fcdc7000640 1 --2- 172.21.15.111:0/341120822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc0107fa0 0x7fcdc0108380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009900 tx=0x7fcdb002a410 comp rx=0 tx=0).stop 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 -- 172.21.15.111:0/341120822 shutdown_connections 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 --2- 172.21.15.111:0/341120822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc010d070 0x7fcdc0114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 --2- 172.21.15.111:0/341120822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc0108950 0x7fcdc010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 --2- 172.21.15.111:0/341120822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc0107fa0 0x7fcdc0108380 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 -- 172.21.15.111:0/341120822 >> 172.21.15.111:0/341120822 conn(0x7fcdc0075cd0 msgr2=0x7fcdc00780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 -- 172.21.15.111:0/341120822 shutdown_connections 2024-04-24T22:43:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.355+0000 7fcdc7000640 1 -- 172.21.15.111:0/341120822 wait complete. 2024-04-24T22:43:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.356+0000 7fcdc7000640 1 Processor -- start 2024-04-24T22:43:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.356+0000 7fcdc7000640 1 -- start start 2024-04-24T22:43:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc0107fa0 0x7fcdc019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc01a2e60 0x7fcdc01a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0118100 con 0x7fcdc010d070 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0117f80 con 0x7fcdc01a2e60 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc7000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0118280 con 0x7fcdc0107fa0 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdb7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdb7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37568/0 (socket says 172.21.15.111:37568) 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdb7400640 1 -- 172.21.15.111:0/77889380 learned_addr learned my addr 172.21.15.111:0/77889380 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc0107fa0 0x7fcdc019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.357+0000 7fcdc5000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc01a2e60 0x7fcdc01a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 -- 172.21.15.111:0/77889380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc0107fa0 msgr2=0x7fcdc019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc0107fa0 0x7fcdc019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 -- 172.21.15.111:0/77889380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc01a2e60 msgr2=0x7fcdc01a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc01a2e60 0x7fcdc01a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 -- 172.21.15.111:0/77889380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc01a3950 con 0x7fcdc010d070 2024-04-24T22:43:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdb7400640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcda800d770 tx=0x7fcda800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdbd600640 1 -- 172.21.15.111:0/77889380 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8017070 con 0x7fcdc010d070 2024-04-24T22:43:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc01a3c40 con 0x7fcdc010d070 2024-04-24T22:43:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdbd600640 1 -- 172.21.15.111:0/77889380 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8009c30 con 0x7fcdc010d070 2024-04-24T22:43:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdbd600640 1 -- 172.21.15.111:0/77889380 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda800f620 con 0x7fcdc010d070 2024-04-24T22:43:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.358+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc01ab430 con 0x7fcdc010d070 2024-04-24T22:43:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.362+0000 7fcdc9e7e740 1 Processor -- start 2024-04-24T22:43:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 -- start start 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdbd600640 1 -- 172.21.15.111:0/77889380 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcda801a070 con 0x7fcdc010d070 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f43aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555c62e9cab0 con 0x555c62f39720 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555c62e45640 con 0x555c62f43fe0 2024-04-24T22:43:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.363+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c62e98a70 con 0x555c62f38e00 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f43aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:43112/0 (socket says 172.21.15.111:43112) 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 -- 172.21.15.111:0/3991844504 learned_addr learned my addr 172.21.15.111:0/3991844504 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdb7400640 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 -- 172.21.15.111:0/3991844504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 msgr2=0x555c62f463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 -- 172.21.15.111:0/3991844504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 msgr2=0x555c62f43aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f43aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 -- 172.21.15.111:0/3991844504 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c62bb1210 con 0x555c62f38e00 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f43aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.364+0000 7fcdc5000640 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009f50 tx=0x7fcdb80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a21868b558bba49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdb5600640 1 -- 172.21.15.111:0/3991844504 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8004d20 con 0x555c62f38e00 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdb5600640 1 -- 172.21.15.111:0/3991844504 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb8004050 con 0x555c62f38e00 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdb5600640 1 -- 172.21.15.111:0/3991844504 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8005280 con 0x555c62f38e00 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdbd600640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 0x7fcda40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdbd600640 1 -- 172.21.15.111:0/77889380 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcda80ceee0 con 0x7fcdc010d070 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.365+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 0x7fcda40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3991844504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 msgr2=0x555c62f391e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009f50 tx=0x7fcdb80118d0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3991844504 shutdown_connections 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f43aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3991844504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f38e00 0x555c62f391e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3991844504 >> 172.21.15.111:0/3991844504 conn(0x555c62f35310 msgr2=0x555c62f356f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3991844504 shutdown_connections 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3991844504 wait complete. 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 0x7fcda40a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009230 tx=0x7fcdb00058d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.366+0000 7fcdc9e7e740 1 -- start start 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 Processor -- start 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f39720 0x555c62f414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555c62f46c10 con 0x555c62f38e00 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555c62e58990 con 0x555c62f43fe0 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c62f46d90 con 0x555c62f39720 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdc9e7e740 10 client.0 _ll_register_callbacks cb 0x555c62e5a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37576/0 (socket says 172.21.15.111:37576) 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.367+0000 7fcdbfe00640 1 -- 172.21.15.111:0/2761092870 learned_addr learned my addr 172.21.15.111:0/2761092870 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdb7400640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f39720 0x555c62f414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdc5000640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 -- 172.21.15.111:0/2761092870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f39720 msgr2=0x555c62f414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f39720 0x555c62f414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 -- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 msgr2=0x555c62f419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 -- 172.21.15.111:0/2761092870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c62c7d620 con 0x555c62f38e00 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdc5000640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009900 tx=0x7fcdb0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dcdbde165cefe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdae000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.368+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb004e070 con 0x555c62f38e00 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcdb0003a00 con 0x555c62f38e00 2024-04-24T22:43:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb8004050 con 0x555c62f38e00 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcdb005a050 con 0x555c62f38e00 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdc9e7e740 10 client.15339 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c62f52ac0 con 0x555c62f38e00 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb8004050 con 0x555c62f38e00 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.369+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcdb004ac90 con 0x555c62f38e00 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 20 client.15339 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62f53f80 0x555c62f56390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 4 client.15339 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c62ea9ff0 con 0x555c62f53f80 2024-04-24T22:43:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdb7400640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62f53f80 0x555c62f56390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdae000640 10 client.15339 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.370+0000 7fcdb7400640 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62f53f80 0x555c62f56390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab475980856f5d86 server_cookie=ca95387aeb0ca4d8 in_seq=0 out_seq=0 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.371+0000 7fcdae000640 1 -- 172.21.15.111:0/2761092870 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c62ea9ff0 con 0x555c62f53f80 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.371+0000 7fcdae000640 10 client.15339 handle_command_reply: tid=0 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.371+0000 7fcdc9e7e740 1 client.15339 shutdown 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.371+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 msgr2=0x555c62f40f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.371+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009900 tx=0x7fcdb0009260 comp rx=0 tx=0).stop 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 shutdown_connections 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62f53f80 0x555c62f56390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f43fe0 0x555c62f419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f39720 0x555c62f414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/2761092870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f40f60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdae000640 0 client.15339 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 >> 172.21.15.111:0/2761092870 conn(0x555c62f35310 msgr2=0x555c62f3b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 shutdown_connections 2024-04-24T22:43:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/2761092870 wait complete. 2024-04-24T22:43:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.372+0000 7fcdc9e7e740 20 client.15339 trim_cache size 0 max 16384 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 Processor -- start 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 -- start start 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f39720 0x555c62f41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b61e0 con 0x555c62f38e00 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b3ab0 con 0x555c62f39720 2024-04-24T22:43:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.388+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b1920 con 0x555c62f43fe0 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37584/0 (socket says 172.21.15.111:37584) 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 -- 172.21.15.111:0/291087863 learned_addr learned my addr 172.21.15.111:0/291087863 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdc5000640 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdb7400640 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f39720 0x555c62f41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 -- 172.21.15.111:0/291087863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 msgr2=0x555c62f5bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f5bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 -- 172.21.15.111:0/291087863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f39720 msgr2=0x555c62f41800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f39720 0x555c62f41800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 -- 172.21.15.111:0/291087863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c62f35140 con 0x555c62f38e00 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.389+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcdb002a930 tx=0x7fcdb0031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7ceb49d2843779 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdb4c00640 1 -- 172.21.15.111:0/291087863 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb004e040 con 0x555c62f38e00 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdb4c00640 1 -- 172.21.15.111:0/291087863 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0063070 con 0x555c62f38e00 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdb4c00640 1 -- 172.21.15.111:0/291087863 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0035040 con 0x555c62f38e00 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/291087863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 msgr2=0x555c62f34c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcdb002a930 tx=0x7fcdb0031a30 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/291087863 shutdown_connections 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f5bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f39720 0x555c62f41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/291087863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f38e00 0x555c62f34c00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/291087863 >> 172.21.15.111:0/291087863 conn(0x555c62f35310 msgr2=0x555c62f3b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/291087863 shutdown_connections 2024-04-24T22:43:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.390+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/291087863 wait complete. 2024-04-24T22:43:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.391+0000 7fcdc9e7e740 1 -- start start 2024-04-24T22:43:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.391+0000 7fcdc9e7e740 1 Processor -- start 2024-04-24T22:43:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.391+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.391+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc9e7e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f48fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b1f70 con 0x555c62f39720 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b33e0 con 0x555c62f38e00 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc9e7e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b2640 con 0x555c62f43fe0 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc9e7e740 10 client.0 _ll_register_callbacks cb 0x7fcdb0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdb7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f48fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43628/0 (socket says 172.21.15.111:43628) 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 -- 172.21.15.111:0/3765770150 learned_addr learned my addr 172.21.15.111:0/3765770150 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 -- 172.21.15.111:0/3765770150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 msgr2=0x555c62f48fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f48fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 -- 172.21.15.111:0/3765770150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 msgr2=0x555c62f4c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f4c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.392+0000 7fcdbfe00640 1 -- 172.21.15.111:0/3765770150 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c62f49650 con 0x555c62f38e00 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdbfe00640 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004750 tx=0x7fcdb0003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dcdb70522208161 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdae000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb006d020 con 0x555c62f38e00 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcdb003d890 con 0x555c62f38e00 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb0063070 con 0x555c62f38e00 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcdb0078050 con 0x555c62f38e00 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdc9e7e740 10 client.25430 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.393+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c62fbaed0 con 0x555c62f38e00 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0063070 con 0x555c62f38e00 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcdb004aa30 con 0x555c62f38e00 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 10 client.25430 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 20 client.25430 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62fbb520 0x555c62fbd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 4 client.25430 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:43:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.394+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c62ea4340 con 0x555c62fbb520 2024-04-24T22:43:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.395+0000 7fcdb7400640 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62fbb520 0x555c62fbd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.395+0000 7fcdb7400640 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62fbb520 0x555c62fbd940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce948c2c3aae6d1b server_cookie=b70581c23900d2a6 in_seq=0 out_seq=0 2024-04-24T22:43:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.395+0000 7fcdae000640 10 client.25430 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdae000640 1 -- 172.21.15.111:0/3765770150 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c62ea4340 con 0x555c62fbb520 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdae000640 10 client.25430 handle_command_reply: tid=0 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 client.25430 shutdown 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 msgr2=0x555c62f4bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004750 tx=0x7fcdb0003a30 comp rx=0 tx=0).stop 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 shutdown_connections 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555c62fbb520 0x555c62fbd940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c62f43fe0 0x555c62f48fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555c62f39720 0x555c62f4c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 --2- 172.21.15.111:0/3765770150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555c62f38e00 0x555c62f4bfe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdae000640 0 client.25430 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 >> 172.21.15.111:0/3765770150 conn(0x555c62f35310 msgr2=0x555c62f42790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 shutdown_connections 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 1 -- 172.21.15.111:0/3765770150 wait complete. 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.409+0000 7fcdc9e7e740 20 client.25430 trim_cache size 0 max 16384 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:43:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 msgr2=0x7fcda40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 0x7fcda40a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009230 tx=0x7fcdb00058d0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 msgr2=0x7fcdc019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcda800d770 tx=0x7fcda800dc30 comp rx=0 tx=0).stop 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 shutdown_connections 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdc01a2e60 0x7fcdc01a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc010d070 0x7fcdc019eae0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcda40a2da0 0x7fcda40a5260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 --2- 172.21.15.111:0/77889380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdc0107fa0 0x7fcdc019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 >> 172.21.15.111:0/77889380 conn(0x7fcdc0075cd0 msgr2=0x7fcdc0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.410+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 shutdown_connections 2024-04-24T22:43:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:45.411+0000 7fcdc7000640 1 -- 172.21.15.111:0/77889380 wait complete. 2024-04-24T22:43:45.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:45 smithi186 ceph-mon[32753]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-04-24T22:43:46.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:45 smithi111 ceph-mon[27746]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-04-24T22:43:46.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:45 smithi171 ceph-mon[32337]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-04-24T22:43:47.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:47 smithi186 ceph-mon[32753]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:43:48.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:47 smithi111 ceph-mon[27746]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:43:48.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:47 smithi171 ceph-mon[32337]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:43:49.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:49 smithi186 ceph-mon[32753]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:50.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:49 smithi111 ceph-mon[27746]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:50.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:49 smithi171 ceph-mon[32337]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-24T22:43:51.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:51 smithi186 ceph-mon[32753]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-24T22:43:52.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:51 smithi111 ceph-mon[27746]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-24T22:43:52.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:51 smithi171 ceph-mon[32337]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-24T22:43:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:53 smithi186 ceph-mon[32753]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-24T22:43:54.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:53 smithi111 ceph-mon[27746]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-24T22:43:54.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:53 smithi171 ceph-mon[32337]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-24T22:43:55.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:55 smithi186 ceph-mon[32753]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:55 smithi111 ceph-mon[27746]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:56.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:55 smithi171 ceph-mon[32337]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:57.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:57 smithi186 ceph-mon[32753]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:58.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:57 smithi111 ceph-mon[27746]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:57 smithi171 ceph-mon[32337]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-24T22:43:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 -- 172.21.15.111:0/3085753758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b4074330 msgr2=0x7f45b410daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 --2- 172.21.15.111:0/3085753758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b4074330 0x7f45b410daf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a002a480 comp rx=0 tx=0).stop 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 -- 172.21.15.111:0/3085753758 shutdown_connections 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 --2- 172.21.15.111:0/3085753758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b410e030 0x7f45b4110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 --2- 172.21.15.111:0/3085753758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b4074330 0x7f45b410daf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 --2- 172.21.15.111:0/3085753758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4073a10 0x7f45b4073df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 -- 172.21.15.111:0/3085753758 >> 172.21.15.111:0/3085753758 conn(0x7f45b4078bd0 msgr2=0x7f45b4078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 -- 172.21.15.111:0/3085753758 shutdown_connections 2024-04-24T22:43:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.550+0000 7f45b9000640 1 -- 172.21.15.111:0/3085753758 wait complete. 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.551+0000 7f45b9000640 1 Processor -- start 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 -- start start 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 0x7f45b419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 0x7f45b419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4113d30 con 0x7f45b410e030 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4113bb0 con 0x7f45b4074330 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b9000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4113eb0 con 0x7f45b4073a10 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 0x7f45b419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45504/0 (socket says 172.21.15.111:45504) 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b2800640 1 -- 172.21.15.111:0/3697411651 learned_addr learned my addr 172.21.15.111:0/3697411651 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.552+0000 7f45b1400640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 0x7f45b419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 -- 172.21.15.111:0/3697411651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 msgr2=0x7f45b419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 0x7f45b419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 -- 172.21.15.111:0/3697411651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 msgr2=0x7f45b419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 0x7f45b419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 -- 172.21.15.111:0/3697411651 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b419f180 con 0x7f45b410e030 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b1400640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 0x7f45b419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b1e00640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 0x7f45b419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b2800640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00eab0 tx=0x7f45ac009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45aaa00640 1 -- 172.21.15.111:0/3697411651 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ce40 con 0x7f45b410e030 2024-04-24T22:43:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45aaa00640 1 -- 172.21.15.111:0/3697411651 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac005020 con 0x7f45b410e030 2024-04-24T22:43:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b419f470 con 0x7f45b410e030 2024-04-24T22:43:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45aaa00640 1 -- 172.21.15.111:0/3697411651 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac010760 con 0x7f45b410e030 2024-04-24T22:43:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.553+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a6d00 con 0x7f45b410e030 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45aaa00640 1 -- 172.21.15.111:0/3697411651 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f45ac004a40 con 0x7f45b410e030 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 Processor -- start 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 -- start start 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a7081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a712aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a712fe0 0x56311a7153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56311a66bab0 con 0x56311a707e00 2024-04-24T22:43:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56311a614640 con 0x56311a708720 2024-04-24T22:43:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.557+0000 7f45bbf78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56311a667a70 con 0x56311a712fe0 2024-04-24T22:43:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a7081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a7081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45508/0 (socket says 172.21.15.111:45508) 2024-04-24T22:43:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1400640 1 -- 172.21.15.111:0/2849960909 learned_addr learned my addr 172.21.15.111:0/2849960909 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a712aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1400640 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a712fe0 0x56311a7153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 -- 172.21.15.111:0/2849960909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a712fe0 msgr2=0x56311a7153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a712fe0 0x56311a7153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 -- 172.21.15.111:0/2849960909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 msgr2=0x56311a7081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a7081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45b1e00640 1 -- 172.21.15.111:0/2849960909 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311a380210 con 0x56311a708720 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.558+0000 7f45aaa00640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 0x7f45980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f45b1e00640 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a712aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009fd0 tx=0x7f45a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bc1cdc092598ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f45aaa00640 1 -- 172.21.15.111:0/3697411651 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f45ac0cfa90 con 0x7f45b410e030 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f459f400640 1 -- 172.21.15.111:0/2849960909 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004030 con 0x56311a708720 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f45b1400640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 0x7f45980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f459f400640 1 -- 172.21.15.111:0/2849960909 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a4005500 con 0x56311a708720 2024-04-24T22:43:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.559+0000 7f459f400640 1 -- 172.21.15.111:0/2849960909 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004cc0 con 0x56311a708720 2024-04-24T22:43:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- 172.21.15.111:0/2849960909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 msgr2=0x56311a712aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45b1400640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 0x7f45980a52e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56311a705730 tx=0x7f45a0005b50 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a712aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009fd0 tx=0x7f45a40118d0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- 172.21.15.111:0/2849960909 shutdown_connections 2024-04-24T22:43:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a712fe0 0x56311a7153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a712aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2849960909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a7081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- 172.21.15.111:0/2849960909 >> 172.21.15.111:0/2849960909 conn(0x56311a704310 msgr2=0x56311a7046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- 172.21.15.111:0/2849960909 shutdown_connections 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- 172.21.15.111:0/2849960909 wait complete. 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.560+0000 7f45bbf78740 1 -- start start 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 Processor -- start 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a710480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a712fe0 0x56311a7109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56311a715c10 con 0x56311a707e00 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56311a627990 con 0x56311a712fe0 2024-04-24T22:43:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56311a715d90 con 0x56311a708720 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45bbf78740 10 client.0 _ll_register_callbacks cb 0x56311a629950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45512/0 (socket says 172.21.15.111:45512) 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.561+0000 7f45b1e00640 1 -- 172.21.15.111:0/3337436431 learned_addr learned my addr 172.21.15.111:0/3337436431 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b2800640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a712fe0 0x56311a7109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1400640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a710480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 -- 172.21.15.111:0/3337436431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 msgr2=0x56311a710480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a710480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 -- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a712fe0 msgr2=0x56311a7109c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a712fe0 0x56311a7109c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 -- 172.21.15.111:0/3337436431 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311a44c620 con 0x56311a707e00 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f45b1e00640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011dd0 tx=0x7f45a400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaf37abb2263f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f4593e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.562+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4029020 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f45a4004bd0 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a4005500 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f45a4034050 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 10 client.15357 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56311a721a80 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a4005500 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f45a4025470 con 0x56311a707e00 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 10 client.15357 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 20 client.15357 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a722ee0 0x56311a7252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 4 client.15357 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:43:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.563+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56311a678ff0 con 0x56311a722ee0 2024-04-24T22:43:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.564+0000 7f45b1400640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a722ee0 0x56311a7252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.565+0000 7f45b1400640 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a722ee0 0x56311a7252d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73990e65bc9e1a4e server_cookie=160ca19cd65a66a7 in_seq=0 out_seq=0 2024-04-24T22:43:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.565+0000 7f4593e00640 10 client.15357 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.565+0000 7f4593e00640 1 -- 172.21.15.111:0/3337436431 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56311a678ff0 con 0x56311a722ee0 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.565+0000 7f4593e00640 10 client.15357 handle_command_reply: tid=0 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 client.15357 shutdown 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 msgr2=0x56311a70ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011dd0 tx=0x7f45a400fc40 comp rx=0 tx=0).stop 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 shutdown_connections 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a722ee0 0x56311a7252d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a712fe0 0x56311a7109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a710480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 --2- 172.21.15.111:0/3337436431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a707e00 0x56311a70ff40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 >> 172.21.15.111:0/3337436431 conn(0x56311a704310 msgr2=0x56311a70a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f4593e00640 0 client.15357 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 shutdown_connections 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 1 -- 172.21.15.111:0/3337436431 wait complete. 2024-04-24T22:43:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.566+0000 7f45bbf78740 20 client.15357 trim_cache size 0 max 16384 2024-04-24T22:43:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.582+0000 7f45bbf78740 1 Processor -- start 2024-04-24T22:43:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.582+0000 7f45bbf78740 1 -- start start 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a707e00 0x56311a71eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a709ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b1ef0 con 0x56311a712fe0 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41af7c0 con 0x56311a707e00 2024-04-24T22:43:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45bbf78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41ad610 con 0x56311a708720 2024-04-24T22:43:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45520/0 (socket says 172.21.15.111:45520) 2024-04-24T22:43:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 -- 172.21.15.111:0/2987345525 learned_addr learned my addr 172.21.15.111:0/2987345525 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b1e00640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a707e00 0x56311a71eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b1400640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a709ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 -- 172.21.15.111:0/2987345525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 msgr2=0x56311a709ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a709ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 -- 172.21.15.111:0/2987345525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a707e00 msgr2=0x56311a71eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a707e00 0x56311a71eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b2800640 1 -- 172.21.15.111:0/2987345525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311a715600 con 0x56311a712fe0 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.583+0000 7f45b1400640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a709ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f45b2800640 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0983e0 tx=0x7f45ac09c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a1a504a02eb22c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f459ea00640 1 -- 172.21.15.111:0/2987345525 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0b4070 con 0x56311a712fe0 2024-04-24T22:43:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f459ea00640 1 -- 172.21.15.111:0/2987345525 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac0b6020 con 0x56311a712fe0 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f459ea00640 1 -- 172.21.15.111:0/2987345525 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0cd430 con 0x56311a712fe0 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f45bbf78740 1 -- 172.21.15.111:0/2987345525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 msgr2=0x56311a72a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0983e0 tx=0x7f45ac09c740 comp rx=0 tx=0).stop 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.584+0000 7f45bbf78740 1 -- 172.21.15.111:0/2987345525 shutdown_connections 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a72a9e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a708720 0x56311a709ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 --2- 172.21.15.111:0/2987345525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a707e00 0x56311a71eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 -- 172.21.15.111:0/2987345525 >> 172.21.15.111:0/2987345525 conn(0x56311a704310 msgr2=0x56311a7097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 -- 172.21.15.111:0/2987345525 shutdown_connections 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 -- 172.21.15.111:0/2987345525 wait complete. 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 -- start start 2024-04-24T22:43:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.585+0000 7f45bbf78740 1 Processor -- start 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 0x56311a71b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a70c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41adc80 con 0x56311a712fe0 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41af0f0 con 0x56311a708720 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41ae350 con 0x56311a707e00 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45bbf78740 10 client.0 _ll_register_callbacks cb 0x7f45a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45524/0 (socket says 172.21.15.111:45524) 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 -- 172.21.15.111:0/1318248186 learned_addr learned my addr 172.21.15.111:0/1318248186 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:43:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b1e00640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 0x56311a71b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b1400640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a70c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 -- 172.21.15.111:0/1318248186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 msgr2=0x56311a71b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 0x56311a71b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 -- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 msgr2=0x56311a70c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a70c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b2800640 1 -- 172.21.15.111:0/1318248186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311a70d490 con 0x56311a712fe0 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b1400640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a70c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.586+0000 7f45b1e00640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 0x56311a71b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f45b2800640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ea80 tx=0x7f45ac0a4bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf20fd3a61b382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f4593e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac07e030 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f45ac0b7040 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac0b6020 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f45ac0b4070 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f45bbf78740 10 client.15363 fetch_fsmap learned FSMap version 24 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56311a7919c0 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.587+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac0b6020 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f45ac0babc0 con 0x56311a712fe0 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 10 client.15363 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 20 client.15363 populate_metadata read hostname 'smithi111' 2024-04-24T22:43:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a792070 0x56311a794460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 4 client.15363 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:43:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56311a673340 con 0x56311a792070 2024-04-24T22:43:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.588+0000 7f45b1e00640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a792070 0x56311a794460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.589+0000 7f45b1e00640 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a792070 0x56311a794460 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab2c76a73bbf0fb server_cookie=558a6d8a9012e7e0 in_seq=0 out_seq=0 2024-04-24T22:43:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.589+0000 7f4593e00640 10 client.15363 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f4593e00640 1 -- 172.21.15.111:0/1318248186 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56311a673340 con 0x56311a792070 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f4593e00640 10 client.15363 handle_command_reply: tid=0 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 client.15363 shutdown 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 msgr2=0x56311a70cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ea80 tx=0x7f45ac0a4bf0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 shutdown_connections 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56311a792070 0x56311a794460 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56311a712fe0 0x56311a70cd50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56311a708720 0x56311a70c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 --2- 172.21.15.111:0/1318248186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56311a707e00 0x56311a71b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f4593e00640 0 client.15363 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.603+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 >> 172.21.15.111:0/1318248186 conn(0x56311a704310 msgr2=0x56311a70a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.604+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 shutdown_connections 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.604+0000 7f45bbf78740 1 -- 172.21.15.111:0/1318248186 wait complete. 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.604+0000 7f45bbf78740 20 client.15363 trim_cache size 0 max 16384 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:43:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.604+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 msgr2=0x7f45980a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 0x7f45980a52e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56311a705730 tx=0x7f45a0005b50 comp rx=0 tx=0).stop 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 msgr2=0x7f45b419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00eab0 tx=0x7f45ac009f80 comp rx=0 tx=0).stop 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 shutdown_connections 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b410e030 0x7f45b419ea50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45b4074330 0x7f45b419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45980a2e20 0x7f45980a52e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 --2- 172.21.15.111:0/3697411651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b4073a10 0x7f45b419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 >> 172.21.15.111:0/3697411651 conn(0x7f45b4078bd0 msgr2=0x7f45b40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 shutdown_connections 2024-04-24T22:43:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:43:58.605+0000 7f45b9000640 1 -- 172.21.15.111:0/3697411651 wait complete. 2024-04-24T22:43:59.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:43:59 smithi186 ceph-mon[32753]: pgmap v64: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-24T22:44:00.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:43:59 smithi111 ceph-mon[27746]: pgmap v64: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-24T22:44:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:43:59 smithi171 ceph-mon[32337]: pgmap v64: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-24T22:44:01.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:01 smithi186 ceph-mon[32753]: pgmap v65: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:44:02.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:01 smithi111 ceph-mon[27746]: pgmap v65: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:44:02.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:01 smithi171 ceph-mon[32337]: pgmap v65: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-24T22:44:03.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:03 smithi186 ceph-mon[32753]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-24T22:44:04.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:03 smithi111 ceph-mon[27746]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-24T22:44:04.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:03 smithi171 ceph-mon[32337]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-24T22:44:05.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:05 smithi186 ceph-mon[32753]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:06.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:05 smithi111 ceph-mon[27746]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:06.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:05 smithi171 ceph-mon[32337]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:07 smithi186 ceph-mon[32753]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:07 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:44:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:07 smithi111 ceph-mon[27746]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:08.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:07 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:44:08.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:07 smithi171 ceph-mon[32337]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-24T22:44:08.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:07 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:44:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:09 smithi186 ceph-mon[32753]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-24T22:44:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:09 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:44:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:09 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:44:10.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:09 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:44:10.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:09 smithi111 ceph-mon[27746]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-24T22:44:10.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:09 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:44:10.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:09 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:44:10.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:09 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:44:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:09 smithi171 ceph-mon[32337]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-24T22:44:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:09 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:44:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:09 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:44:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:09 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 -- 172.21.15.111:0/3997871560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38101f30 msgr2=0x7f0d3810e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 --2- 172.21.15.111:0/3997871560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38101f30 0x7f0d3810e3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ca0 tx=0x7f0d3002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 -- 172.21.15.111:0/3997871560 shutdown_connections 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 --2- 172.21.15.111:0/3997871560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38101f30 0x7f0d3810e3f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 --2- 172.21.15.111:0/3997871560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101570 0x7f0d381019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 --2- 172.21.15.111:0/3997871560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38107630 0x7f0d38107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 -- 172.21.15.111:0/3997871560 >> 172.21.15.111:0/3997871560 conn(0x7f0d380fd080 msgr2=0x7f0d380ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 -- 172.21.15.111:0/3997871560 shutdown_connections 2024-04-24T22:44:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.749+0000 7f0d3f000640 1 -- 172.21.15.111:0/3997871560 wait complete. 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.750+0000 7f0d3f000640 1 Processor -- start 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.750+0000 7f0d3f000640 1 -- start start 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.750+0000 7f0d3f000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 0x7f0d38073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.750+0000 7f0d3f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3f000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38107630 0x7f0d380744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3f000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38111940 con 0x7f0d38107630 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381117c0 con 0x7f0d38101f30 2024-04-24T22:44:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3f000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38111ac0 con 0x7f0d38101570 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 0x7f0d38073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33304/0 (socket says 172.21.15.111:33304) 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3d200640 1 -- 172.21.15.111:0/1762028943 learned_addr learned my addr 172.21.15.111:0/1762028943 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3e600640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38107630 0x7f0d380744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1762028943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 msgr2=0x7f0d38073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38107630 msgr2=0x7f0d380744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38107630 0x7f0d380744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1762028943 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d381a6220 con 0x7f0d38101570 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.751+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 0x7f0d38073a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d36000640 1 -- 172.21.15.111:0/1762028943 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00cd80 con 0x7f0d38101570 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d36000640 1 -- 172.21.15.111:0/1762028943 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c0035f0 con 0x7f0d38101570 2024-04-24T22:44:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d381a6510 con 0x7f0d38101570 2024-04-24T22:44:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d36000640 1 -- 172.21.15.111:0/1762028943 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c010690 con 0x7f0d38101570 2024-04-24T22:44:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.752+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381a6a60 con 0x7f0d38101570 2024-04-24T22:44:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.755+0000 7f0d36000640 1 -- 172.21.15.111:0/1762028943 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0d2c003b30 con 0x7f0d38101570 2024-04-24T22:44:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 Processor -- start 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 -- start start 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f382390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55844f2e5ab0 con 0x55844f38d190 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55844f28e640 con 0x55844f381fb0 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d433eb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55844f2e1a70 con 0x55844f3828d0 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d3d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d3d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38332/0 (socket says 172.21.15.111:38332) 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d3d200640 1 -- 172.21.15.111:0/1679168221 learned_addr learned my addr 172.21.15.111:0/1679168221 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.756+0000 7f0d3e600640 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f382390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d36000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 0x7f0d200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d36000640 1 -- 172.21.15.111:0/1762028943 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f0d2c0cf700 con 0x7f0d38101570 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 -- 172.21.15.111:0/1679168221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 msgr2=0x55844f38cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 -- 172.21.15.111:0/1679168221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 msgr2=0x55844f382390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3e600640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 0x7f0d200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f382390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 -- 172.21.15.111:0/1679168221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844effa210 con 0x55844f38d190 2024-04-24T22:44:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.757+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d240115f0 tx=0x7f0d24011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c0f0a7ecec471a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d2aa00640 1 -- 172.21.15.111:0/1679168221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2401e070 con 0x55844f38d190 2024-04-24T22:44:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d2aa00640 1 -- 172.21.15.111:0/1679168221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2400f660 con 0x55844f38d190 2024-04-24T22:44:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d2aa00640 1 -- 172.21.15.111:0/1679168221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240038d0 con 0x55844f38d190 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 -- 172.21.15.111:0/1679168221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 msgr2=0x55844f38f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d3e600640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 0x7f0d200a51b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d30006010 tx=0x7f0d3002a8c0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d240115f0 tx=0x7f0d24011ab0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 -- 172.21.15.111:0/1679168221 shutdown_connections 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38f580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1679168221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f382390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.758+0000 7f0d433eb740 1 -- 172.21.15.111:0/1679168221 >> 172.21.15.111:0/1679168221 conn(0x55844f37e4c0 msgr2=0x55844f37e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.759+0000 7f0d433eb740 1 -- 172.21.15.111:0/1679168221 shutdown_connections 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.759+0000 7f0d433eb740 1 -- 172.21.15.111:0/1679168221 wait complete. 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.759+0000 7f0d433eb740 1 -- start start 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.759+0000 7f0d433eb740 1 Processor -- start 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f38a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55844f38fdc0 con 0x55844f38d190 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55844f2a1990 con 0x55844f381fb0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55844f38ff40 con 0x55844f3828d0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d433eb740 10 client.0 _ll_register_callbacks cb 0x55844f2a3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f38a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3e600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f38a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33322/0 (socket says 172.21.15.111:33322) 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3dc00640 1 -- 172.21.15.111:0/198239678 learned_addr learned my addr 172.21.15.111:0/198239678 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 -- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 msgr2=0x55844f38a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f38a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 -- 172.21.15.111:0/198239678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 msgr2=0x55844f38ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.760+0000 7f0d3d200640 1 -- 172.21.15.111:0/198239678 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844f0c6620 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d3d200640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38a5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d2401c040 tx=0x7f0d24005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58473a6c777dca03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d1f400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2401e040 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f0d24005460 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2400f660 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d24030050 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d433eb740 10 client.25480 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.761+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55844f39bc10 con 0x55844f3828d0 2024-04-24T22:44:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2400f660 con 0x55844f3828d0 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0d24021070 con 0x55844f3828d0 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 10 client.25480 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 20 client.25480 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f39d010 0x55844f39f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 4 client.25480 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.762+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55844f2f2ff0 con 0x55844f39d010 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.763+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f39d010 0x55844f39f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.763+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f39d010 0x55844f39f400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6f984ed13327cb server_cookie=40353addcc83e6b in_seq=0 out_seq=0 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.763+0000 7f0d1f400640 10 client.25480 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d1f400640 1 -- 172.21.15.111:0/198239678 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55844f2f2ff0 con 0x55844f39d010 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d1f400640 10 client.25480 handle_command_reply: tid=0 2024-04-24T22:44:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 client.25480 shutdown 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 msgr2=0x55844f38a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38a5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d2401c040 tx=0x7f0d24005270 comp rx=0 tx=0).stop 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 shutdown_connections 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f39d010 0x55844f39f400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f38ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f38a5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 --2- 172.21.15.111:0/198239678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f381fb0 0x55844f38a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d1f400640 0 client.25480 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 >> 172.21.15.111:0/198239678 conn(0x55844f37e4c0 msgr2=0x55844f3841f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 shutdown_connections 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 1 -- 172.21.15.111:0/198239678 wait complete. 2024-04-24T22:44:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.764+0000 7f0d433eb740 20 client.25480 trim_cache size 0 max 16384 2024-04-24T22:44:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.780+0000 7f0d433eb740 1 Processor -- start 2024-04-24T22:44:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.780+0000 7f0d433eb740 1 -- start start 2024-04-24T22:44:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f37dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 0x55844f37e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b1cc0 con 0x55844f381fb0 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381af590 con 0x55844f38d190 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d433eb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ad3e0 con 0x55844f3828d0 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38342/0 (socket says 172.21.15.111:38342) 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 -- 172.21.15.111:0/2552402711 learned_addr learned my addr 172.21.15.111:0/2552402711 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3d200640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f37dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3e600640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 0x55844f37e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 -- 172.21.15.111:0/2552402711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 msgr2=0x55844f37dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f37dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 -- 172.21.15.111:0/2552402711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 msgr2=0x55844f37e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 0x55844f37e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3dc00640 1 -- 172.21.15.111:0/2552402711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844f389050 con 0x55844f381fb0 2024-04-24T22:44:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.781+0000 7f0d3e600640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 0x55844f37e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c09bb80 tx=0x7f0d2c097fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b326121b44481367 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d2a000640 1 -- 172.21.15.111:0/2552402711 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c09ffa0 con 0x55844f381fb0 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d2a000640 1 -- 172.21.15.111:0/2552402711 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c0b2070 con 0x55844f381fb0 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d2a000640 1 -- 172.21.15.111:0/2552402711 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0cd0a0 con 0x55844f381fb0 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d433eb740 1 -- 172.21.15.111:0/2552402711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 msgr2=0x55844f398c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.782+0000 7f0d433eb740 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c09bb80 tx=0x7f0d2c097fc0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 -- 172.21.15.111:0/2552402711 shutdown_connections 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f38d190 0x55844f37e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f3828d0 0x55844f37dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 --2- 172.21.15.111:0/2552402711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f381fb0 0x55844f398c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 -- 172.21.15.111:0/2552402711 >> 172.21.15.111:0/2552402711 conn(0x55844f37e4c0 msgr2=0x55844f3841f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 -- 172.21.15.111:0/2552402711 shutdown_connections 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 -- 172.21.15.111:0/2552402711 wait complete. 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 -- start start 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.783+0000 7f0d433eb740 1 Processor -- start 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 0x55844f386c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 0x55844f3871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f3876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ada50 con 0x55844f38d190 2024-04-24T22:44:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381aeec0 con 0x55844f3828d0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ae120 con 0x55844f381fb0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d433eb740 10 client.0 _ll_register_callbacks cb 0x7f0d2401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3e600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f3876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 0x55844f3871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3dc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 0x55844f386c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3e600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f3876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38346/0 (socket says 172.21.15.111:38346) 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3e600640 1 -- 172.21.15.111:0/1132570045 learned_addr learned my addr 172.21.15.111:0/1132570045 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 msgr2=0x55844f3871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 0x55844f3871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1132570045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 msgr2=0x55844f3876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.784+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f3876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d3dc00640 1 -- 172.21.15.111:0/1132570045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844f4023b0 con 0x55844f381fb0 2024-04-24T22:44:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 0x55844f3871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d3dc00640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 0x55844f386c70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55844f28e710 tx=0x7f0d2c09ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b34bb02d8f3029d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d1f400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c098260 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f0d2c0cd0a0 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2c0b2070 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c098400 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d433eb740 10 client.25486 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55844f4035c0 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.785+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c0b2070 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0d2c09b120 con 0x55844f381fb0 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 10 client.25486 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 20 client.25486 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f403c30 0x55844f406050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 4 client.25486 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:44:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55844f2ed340 con 0x55844f403c30 2024-04-24T22:44:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f403c30 0x55844f406050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d3d200640 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f403c30 0x55844f406050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923a3c9611dbfc6 server_cookie=b7751767f2d5c56f in_seq=0 out_seq=0 2024-04-24T22:44:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.786+0000 7f0d1f400640 10 client.25486 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d1f400640 1 -- 172.21.15.111:0/1132570045 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55844f2ed340 con 0x55844f403c30 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d1f400640 10 client.25486 handle_command_reply: tid=0 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 client.25486 shutdown 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 msgr2=0x55844f386c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 0x55844f386c70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55844f28e710 tx=0x7f0d2c09ffa0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 shutdown_connections 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55844f403c30 0x55844f406050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55844f38d190 0x55844f3876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55844f3828d0 0x55844f3871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 --2- 172.21.15.111:0/1132570045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55844f381fb0 0x55844f386c70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d1f400640 0 client.25486 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.800+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 >> 172.21.15.111:0/1132570045 conn(0x55844f37e4c0 msgr2=0x55844f3841f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.801+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 shutdown_connections 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.801+0000 7f0d433eb740 1 -- 172.21.15.111:0/1132570045 wait complete. 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.801+0000 7f0d433eb740 20 client.25486 trim_cache size 0 max 16384 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:44:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:44:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 msgr2=0x7f0d200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 0x7f0d200a51b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d30006010 tx=0x7f0d3002a8c0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 msgr2=0x7f0d38073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 0x7f0d38073a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 shutdown_connections 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d38107630 0x7f0d380744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d38101f30 0x7f0d38073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0d200a2cf0 0x7f0d200a51b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 --2- 172.21.15.111:0/1762028943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38101570 0x7f0d38073a40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 >> 172.21.15.111:0/1762028943 conn(0x7f0d380fd080 msgr2=0x7f0d380fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 shutdown_connections 2024-04-24T22:44:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:11.802+0000 7f0d3f000640 1 -- 172.21.15.111:0/1762028943 wait complete. 2024-04-24T22:44:11.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:11 smithi186 ceph-mon[32753]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-24T22:44:12.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:11 smithi111 ceph-mon[27746]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-24T22:44:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:11 smithi171 ceph-mon[32337]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-24T22:44:13.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:13 smithi186 ceph-mon[32753]: pgmap v71: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-04-24T22:44:14.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:13 smithi111 ceph-mon[27746]: pgmap v71: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-04-24T22:44:14.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:13 smithi171 ceph-mon[32337]: pgmap v71: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-04-24T22:44:15.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:15 smithi186 ceph-mon[32753]: pgmap v72: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:16.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:15 smithi111 ceph-mon[27746]: pgmap v72: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:16.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:15 smithi171 ceph-mon[32337]: pgmap v72: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:18.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:17 smithi111 ceph-mon[27746]: pgmap v73: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:18.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:17 smithi171 ceph-mon[32337]: pgmap v73: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:18.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:17 smithi186 ceph-mon[32753]: pgmap v73: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:20.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:19 smithi111 ceph-mon[27746]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:20.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:19 smithi171 ceph-mon[32337]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:19 smithi186 ceph-mon[32753]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:22.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:21 smithi111 ceph-mon[27746]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:22.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:21 smithi171 ceph-mon[32337]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:22.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:21 smithi186 ceph-mon[32753]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:1/600: open fd = 3 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:2/600: open fd = 3 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:3/600: open fd = 3 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.028 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:4/600: open fd = 3 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:5/600: open fd = 3 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:6/600: open fd = 3 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.029 INFO:tasks.workunit.client.0.smithi111.stdout:7/600: open fd = 3 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:8/600: open fd = 3 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.030 INFO:tasks.workunit.client.0.smithi111.stdout:9/600: open fd = 3 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:10/600: open fd = 3 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.034 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:11/600: open fd = 3 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:12/600: open fd = 3 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:13/600: open fd = 3 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:14/600: open fd = 3 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.035 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:15/600: open fd = 3 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:16/600: open fd = 3 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:17/600: open fd = 3 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:18/600: open fd = 3 2024-04-24T22:44:23.036 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:19/600: open fd = 3 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:20/600: open fd = 3 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:21/600: open fd = 3 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:22/600: open fd = 3 2024-04-24T22:44:23.037 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:23/600: open fd = 3 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:24/600: open fd = 3 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:25/600: open fd = 3 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.038 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:26/600: open fd = 3 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:27/600: open fd = 3 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:28/600: open fd = 3 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:29/600: open fd = 3 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.039 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:30/600: open fd = 3 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:31/600: open fd = 3 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:32/600: open fd = 3 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:33/600: open fd = 3 2024-04-24T22:44:23.040 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:34/600: open fd = 3 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:35/600: open fd = 3 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:36/600: open fd = 3 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:37/600: open fd = 3 2024-04-24T22:44:23.041 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:38/600: open fd = 3 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:39/600: open fd = 3 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:40/600: open fd = 3 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.042 INFO:tasks.workunit.client.0.smithi111.stdout:41/600: open fd = 3 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:42/600: open fd = 3 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:43/600: open fd = 3 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.043 INFO:tasks.workunit.client.0.smithi111.stdout:44/600: open fd = 3 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:45/600: open fd = 3 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:46/600: open fd = 3 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:47/600: open fd = 3 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.044 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:48/600: open fd = 3 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:49/600: open fd = 3 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:50/600: open fd = 3 2024-04-24T22:44:23.045 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:51/600: open fd = 3 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:52/600: open fd = 3 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:53/600: open fd = 3 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.046 INFO:tasks.workunit.client.0.smithi111.stdout:54/600: open fd = 3 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:55/600: open fd = 3 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:56/600: open fd = 3 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:57/600: open fd = 3 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.047 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:58/600: open fd = 3 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:59/600: open fd = 3 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:60/600: open fd = 3 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:61/600: open fd = 3 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.048 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:62/600: open fd = 3 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:63/600: open fd = 3 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:44:23.049 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:44:24.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:23 smithi111 ceph-mon[27746]: pgmap v76: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:24.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:23 smithi171 ceph-mon[32337]: pgmap v76: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:23 smithi186 ceph-mon[32753]: pgmap v76: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 -- 172.21.15.111:0/2368300399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c104300 msgr2=0x7fbe6c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 --2- 172.21.15.111:0/2368300399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c104300 0x7fbe6c110810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009cd0 tx=0x7fbe6402a8f0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 -- 172.21.15.111:0/2368300399 shutdown_connections 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 --2- 172.21.15.111:0/2368300399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c104300 0x7fbe6c110810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 --2- 172.21.15.111:0/2368300399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c103940 0x7fbe6c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 --2- 172.21.15.111:0/2368300399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 -- 172.21.15.111:0/2368300399 >> 172.21.15.111:0/2368300399 conn(0x7fbe6c0ff470 msgr2=0x7fbe6c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 -- 172.21.15.111:0/2368300399 shutdown_connections 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.944+0000 7fbe70a00640 1 -- 172.21.15.111:0/2368300399 wait complete. 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.945+0000 7fbe70a00640 1 Processor -- start 2024-04-24T22:44:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 -- start start 2024-04-24T22:44:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c103940 0x7fbe6c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c104300 0x7fbe6c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c113d80 con 0x7fbe6c109950 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c113c00 con 0x7fbe6c104300 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe70a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c113f00 con 0x7fbe6c103940 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe69e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c103940 0x7fbe6c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41818/0 (socket says 172.21.15.111:41818) 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe6a800640 1 -- 172.21.15.111:0/3665627712 learned_addr learned my addr 172.21.15.111:0/3665627712 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.946+0000 7fbe63e00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c104300 0x7fbe6c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 -- 172.21.15.111:0/3665627712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c103940 msgr2=0x7fbe6c19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c103940 0x7fbe6c19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 -- 172.21.15.111:0/3665627712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c104300 msgr2=0x7fbe6c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c104300 0x7fbe6c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 -- 172.21.15.111:0/3665627712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c19f1c0 con 0x7fbe6c109950 2024-04-24T22:44:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe6a800640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005e30 tx=0x7fbe64003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.947+0000 7fbe62a00640 1 -- 172.21.15.111:0/3665627712 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64030dc0 con 0x7fbe6c109950 2024-04-24T22:44:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.948+0000 7fbe62a00640 1 -- 172.21.15.111:0/3665627712 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe64040070 con 0x7fbe6c109950 2024-04-24T22:44:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.948+0000 7fbe62a00640 1 -- 172.21.15.111:0/3665627712 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64045ad0 con 0x7fbe6c109950 2024-04-24T22:44:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.948+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c19f450 con 0x7fbe6c109950 2024-04-24T22:44:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.948+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c1a6d40 con 0x7fbe6c109950 2024-04-24T22:44:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.952+0000 7fbe737a0740 1 Processor -- start 2024-04-24T22:44:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.952+0000 7fbe62a00640 1 -- 172.21.15.111:0/3665627712 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbe64004700 con 0x7fbe6c109950 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.952+0000 7fbe737a0740 1 -- start start 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775be00 0x55577775c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777766aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x5557777693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5557776bfab0 con 0x555777766fe0 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555777668640 con 0x55577775be00 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe737a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5557776bba70 con 0x55577775c720 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x5557777693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x5557777693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41826/0 (socket says 172.21.15.111:41826) 2024-04-24T22:44:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe63e00640 1 -- 172.21.15.111:0/710829321 learned_addr learned my addr 172.21.15.111:0/710829321 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe69e00640 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777766aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe63e00640 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775be00 0x55577775c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.953+0000 7fbe62a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 0x7fbe4c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 -- 172.21.15.111:0/710829321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775be00 msgr2=0x55577775c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775be00 0x55577775c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 -- 172.21.15.111:0/710829321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 msgr2=0x5557777693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x5557777693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 -- 172.21.15.111:0/710829321 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557773d4210 con 0x55577775c720 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe62a00640 1 -- 172.21.15.111:0/3665627712 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbe640f0320 con 0x7fbe6c109950 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe6a800640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 0x7fbe4c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe69e00640 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777766aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009c20 tx=0x7fbe5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfe6c659305f411 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe5b000640 1 -- 172.21.15.111:0/710829321 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004050 con 0x55577775c720 2024-04-24T22:44:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe5b000640 1 -- 172.21.15.111:0/710829321 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c005500 con 0x55577775c720 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.954+0000 7fbe5b000640 1 -- 172.21.15.111:0/710829321 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004d00 con 0x55577775c720 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 -- 172.21.15.111:0/710829321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 msgr2=0x555777766aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe6a800640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 0x7fbe4c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b9380 tx=0x7fbe6402a8f0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777766aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009c20 tx=0x7fbe5c011810 comp rx=0 tx=0).stop 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 -- 172.21.15.111:0/710829321 shutdown_connections 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x5557777693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777766aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 --2- 172.21.15.111:0/710829321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775be00 0x55577775c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 -- 172.21.15.111:0/710829321 >> 172.21.15.111:0/710829321 conn(0x555777758310 msgr2=0x5557777586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 -- 172.21.15.111:0/710829321 shutdown_connections 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.955+0000 7fbe737a0740 1 -- 172.21.15.111:0/710829321 wait complete. 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 -- start start 2024-04-24T22:44:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 Processor -- start 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777764480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555777766fe0 0x5557777649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555777769c10 con 0x55577775be00 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55577767b990 con 0x555777766fe0 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.956+0000 7fbe737a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555777769d90 con 0x55577775c720 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe737a0740 10 client.0 _ll_register_callbacks cb 0x55577767d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41842/0 (socket says 172.21.15.111:41842) 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 -- 172.21.15.111:0/685978862 learned_addr learned my addr 172.21.15.111:0/685978862 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555777766fe0 0x5557777649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe69e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777764480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 -- 172.21.15.111:0/685978862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 msgr2=0x555777764480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777764480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 -- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555777766fe0 msgr2=0x5557777649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555777766fe0 0x5557777649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe63e00640 1 -- 172.21.15.111:0/685978862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557774a0620 con 0x55577775be00 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.957+0000 7fbe69e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777764480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe63e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004410 tx=0x7fbe54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627190d691f34eba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe53400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54003e80 con 0x55577775be00 2024-04-24T22:44:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbe54009040 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c005500 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbe54012030 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe737a0740 10 client.15393 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555777775bd0 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.958+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c005500 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbe540047a0 con 0x55577775be00 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 10 client.15393 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 20 client.15393 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777770e0 0x5557777794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 4 client.15393 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557776ccff0 con 0x5557777770e0 2024-04-24T22:44:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.959+0000 7fbe69e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777770e0 0x5557777794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.960+0000 7fbe69e00640 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777770e0 0x5557777794d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32881c22c1bec240 server_cookie=f6d07d3a885cf1b7 in_seq=0 out_seq=0 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.960+0000 7fbe53400640 10 client.15393 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe53400640 1 -- 172.21.15.111:0/685978862 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557776ccff0 con 0x5557777770e0 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe53400640 10 client.15393 handle_command_reply: tid=0 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 client.15393 shutdown 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 msgr2=0x555777763f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004410 tx=0x7fbe54004730 comp rx=0 tx=0).stop 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 shutdown_connections 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777770e0 0x5557777794d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555777766fe0 0x5557777649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775c720 0x555777764480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 --2- 172.21.15.111:0/685978862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x555777763f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe53400640 0 client.15393 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 >> 172.21.15.111:0/685978862 conn(0x555777758310 msgr2=0x55577775e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 shutdown_connections 2024-04-24T22:44:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 1 -- 172.21.15.111:0/685978862 wait complete. 2024-04-24T22:44:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.961+0000 7fbe737a0740 20 client.15393 trim_cache size 0 max 16384 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.977+0000 7fbe737a0740 1 Processor -- start 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.977+0000 7fbe737a0740 1 -- start start 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.977+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775be00 0x555777772b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.977+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x555777757c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe737a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1b1d60 con 0x555777766fe0 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe737a0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1af630 con 0x55577775c720 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe737a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ad480 con 0x55577775be00 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41846/0 (socket says 172.21.15.111:41846) 2024-04-24T22:44:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 -- 172.21.15.111:0/1661457149 learned_addr learned my addr 172.21.15.111:0/1661457149 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775be00 0x555777772b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe69e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x555777757c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 -- 172.21.15.111:0/1661457149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775be00 msgr2=0x555777772b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775be00 0x555777772b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 -- 172.21.15.111:0/1661457149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 msgr2=0x555777757c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x555777757c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe63e00640 1 -- 172.21.15.111:0/1661457149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555777779840 con 0x555777766fe0 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.978+0000 7fbe69e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x555777757c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x555777766d10 tx=0x7fbe54001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ced196230892a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe5a600640 1 -- 172.21.15.111:0/1661457149 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5402d030 con 0x555777766fe0 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe5a600640 1 -- 172.21.15.111:0/1661457149 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5402e040 con 0x555777766fe0 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe5a600640 1 -- 172.21.15.111:0/1661457149 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540121d0 con 0x555777766fe0 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 -- 172.21.15.111:0/1661457149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 msgr2=0x55577775dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x555777766d10 tx=0x7fbe54001e70 comp rx=0 tx=0).stop 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 -- 172.21.15.111:0/1661457149 shutdown_connections 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555777766fe0 0x55577775dec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x555777757c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.979+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1661457149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55577775be00 0x555777772b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.980+0000 7fbe737a0740 1 -- 172.21.15.111:0/1661457149 >> 172.21.15.111:0/1661457149 conn(0x555777758310 msgr2=0x55577775d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.980+0000 7fbe737a0740 1 -- 172.21.15.111:0/1661457149 shutdown_connections 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.980+0000 7fbe737a0740 1 -- 172.21.15.111:0/1661457149 wait complete. 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.980+0000 7fbe737a0740 1 -- start start 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.980+0000 7fbe737a0740 1 Processor -- start 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x55577776ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 0x55577776b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1adaf0 con 0x55577775be00 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1aef60 con 0x55577775c720 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ae1c0 con 0x555777766fe0 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe737a0740 10 client.0 _ll_register_callbacks cb 0x7fbe54012b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:34148/0 (socket says 172.21.15.111:34148) 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 -- 172.21.15.111:0/1917031796 learned_addr learned my addr 172.21.15.111:0/1917031796 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x55577776ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 0x55577776b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 -- 172.21.15.111:0/1917031796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 msgr2=0x55577776b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 0x55577776b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 -- 172.21.15.111:0/1917031796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 msgr2=0x55577776ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x55577776ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe69e00640 1 -- 172.21.15.111:0/1917031796 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557777e4f30 con 0x55577775c720 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.981+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 0x55577776b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe69e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c004950 tx=0x7fbe5c005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ff1bc712f59528 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe53400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:44:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c031070 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbe5c02ad50 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5402e040 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c03d050 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe737a0740 10 client.25484 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.982+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5557777e6270 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5402e040 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbe5c0035f0 con 0x55577775c720 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 10 client.25484 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 20 client.25484 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777e6830 0x5557777e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 4 client.25484 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557776c7340 con 0x5557777e6830 2024-04-24T22:44:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.983+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777e6830 0x5557777e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.984+0000 7fbe63e00640 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777e6830 0x5557777e8c50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe9e2b1889e16d9 server_cookie=ce1fb4884e8f6ce1 in_seq=0 out_seq=0 2024-04-24T22:44:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.984+0000 7fbe53400640 10 client.25484 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.998+0000 7fbe53400640 1 -- 172.21.15.111:0/1917031796 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557776c7340 con 0x5557777e6830 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.998+0000 7fbe53400640 10 client.25484 handle_command_reply: tid=0 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.998+0000 7fbe737a0740 1 client.25484 shutdown 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 msgr2=0x55577776b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c004950 tx=0x7fbe5c005f50 comp rx=0 tx=0).stop 2024-04-24T22:44:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 shutdown_connections 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5557777e6830 0x5557777e8c50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555777766fe0 0x55577776b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55577775c720 0x55577776b040 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe53400640 0 client.25484 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 --2- 172.21.15.111:0/1917031796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55577775be00 0x55577776ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 >> 172.21.15.111:0/1917031796 conn(0x555777758310 msgr2=0x555777763f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 shutdown_connections 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 1 -- 172.21.15.111:0/1917031796 wait complete. 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:24.999+0000 7fbe737a0740 20 client.25484 trim_cache size 0 max 16384 2024-04-24T22:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:44:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:44:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:44:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:44:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 msgr2=0x7fbe4c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 0x7fbe4c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b9380 tx=0x7fbe6402a8f0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 msgr2=0x7fbe6c19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005e30 tx=0x7fbe64003f10 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 shutdown_connections 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe6c109950 0x7fbe6c19ea90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe6c104300 0x7fbe6c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbe4c0a2da0 0x7fbe4c0a5260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 --2- 172.21.15.111:0/3665627712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe6c103940 0x7fbe6c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 >> 172.21.15.111:0/3665627712 conn(0x7fbe6c0ff470 msgr2=0x7fbe6c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 shutdown_connections 2024-04-24T22:44:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:25.000+0000 7fbe70a00640 1 -- 172.21.15.111:0/3665627712 wait complete. 2024-04-24T22:44:26.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:25 smithi111 ceph-mon[27746]: pgmap v77: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:26.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:25 smithi171 ceph-mon[32337]: pgmap v77: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:25 smithi186 ceph-mon[32753]: pgmap v77: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:28.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:27 smithi111 ceph-mon[27746]: pgmap v78: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:28.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:27 smithi171 ceph-mon[32337]: pgmap v78: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:28.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:27 smithi186 ceph-mon[32753]: pgmap v78: 97 pgs: 97 active+clean; 2.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-04-24T22:44:30.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:29 smithi111 ceph-mon[27746]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:30.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:29 smithi171 ceph-mon[32337]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:30.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:29 smithi186 ceph-mon[32753]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:32.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:31 smithi171 ceph-mon[32337]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:32.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:31 smithi186 ceph-mon[32753]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:32.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:31 smithi111 ceph-mon[27746]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:34.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:33 smithi171 ceph-mon[32337]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-24T22:44:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:33 smithi186 ceph-mon[32753]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-24T22:44:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:33 smithi111 ceph-mon[27746]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-24T22:44:36.052 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:35 smithi186 ceph-mon[32753]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:36.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:35 smithi171 ceph-mon[32337]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:35 smithi111 ceph-mon[27746]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 -- 172.21.15.111:0/3550265176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 msgr2=0x7fe6d8108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 --2- 172.21.15.111:0/3550265176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d8108380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802a400 comp rx=0 tx=0).stop 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 -- 172.21.15.111:0/3550265176 shutdown_connections 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 --2- 172.21.15.111:0/3550265176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d810d070 0x7fe6d8114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 --2- 172.21.15.111:0/3550265176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d8108950 0x7fe6d810c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 --2- 172.21.15.111:0/3550265176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d8108380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.141+0000 7fe6dd400640 1 -- 172.21.15.111:0/3550265176 >> 172.21.15.111:0/3550265176 conn(0x7fe6d8075cd0 msgr2=0x7fe6d80780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.142+0000 7fe6dd400640 1 -- 172.21.15.111:0/3550265176 shutdown_connections 2024-04-24T22:44:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.142+0000 7fe6dd400640 1 -- 172.21.15.111:0/3550265176 wait complete. 2024-04-24T22:44:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 Processor -- start 2024-04-24T22:44:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 -- start start 2024-04-24T22:44:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d81a2e60 0x7fe6d81a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8118170 con 0x7fe6d81a2e60 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8117ff0 con 0x7fe6d810d070 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.143+0000 7fe6dd400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81182f0 con 0x7fe6d8107fa0 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d81a2e60 0x7fe6d81a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:56230/0 (socket says 172.21.15.111:56230) 2024-04-24T22:44:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 -- 172.21.15.111:0/1958206987 learned_addr learned my addr 172.21.15.111:0/1958206987 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 -- 172.21.15.111:0/1958206987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 msgr2=0x7fe6d819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 -- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d81a2e60 msgr2=0x7fe6d81a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d81a2e60 0x7fe6d81a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 -- 172.21.15.111:0/1958206987 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a3950 con 0x7fe6d810d070 2024-04-24T22:44:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.144+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e9f0 tx=0x7fe6c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.145+0000 7fe6cf400640 1 -- 172.21.15.111:0/1958206987 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ccc0 con 0x7fe6d810d070 2024-04-24T22:44:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.145+0000 7fe6cf400640 1 -- 172.21.15.111:0/1958206987 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0005500 con 0x7fe6d810d070 2024-04-24T22:44:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.145+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a3c40 con 0x7fe6d810d070 2024-04-24T22:44:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.145+0000 7fe6cf400640 1 -- 172.21.15.111:0/1958206987 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0010690 con 0x7fe6d810d070 2024-04-24T22:44:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.145+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81ab430 con 0x7fe6d810d070 2024-04-24T22:44:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.148+0000 7fe6e0247740 1 Processor -- start 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.148+0000 7fe6cf400640 1 -- 172.21.15.111:0/1958206987 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe6c00046c0 con 0x7fe6d810d070 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.148+0000 7fe6e0247740 1 -- start start 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6d851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6d8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6ce8ab0 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6c91640 con 0x55fdb6d85720 2024-04-24T22:44:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6e0247740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6ce4a70 con 0x55fdb6d84e00 2024-04-24T22:44:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6d8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47506/0 (socket says 172.21.15.111:47506) 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6cfe00640 1 -- 172.21.15.111:0/1910411784 learned_addr learned my addr 172.21.15.111:0/1910411784 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.149+0000 7fe6d7200640 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6d851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 -- 172.21.15.111:0/1910411784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 msgr2=0x55fdb6d851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6d851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 -- 172.21.15.111:0/1910411784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 msgr2=0x55fdb6d923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 -- 172.21.15.111:0/1910411784 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb69fd210 con 0x55fdb6d85720 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d6800640 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6d8faa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802a9f0 tx=0x7fe6c8005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a19fb1ed703d063 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6cf400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 0x7fe6bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6cf400640 1 -- 172.21.15.111:0/1958206987 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fe6c00cf6f0 con 0x7fe6d810d070 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6d7200640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 0x7fe6bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6c7400640 1 -- 172.21.15.111:0/1910411784 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035040 con 0x55fdb6d85720 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6c7400640 1 -- 172.21.15.111:0/1910411784 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004400 con 0x55fdb6d85720 2024-04-24T22:44:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.150+0000 7fe6c7400640 1 -- 172.21.15.111:0/1910411784 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80035f0 con 0x55fdb6d85720 2024-04-24T22:44:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 -- 172.21.15.111:0/1910411784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 msgr2=0x55fdb6d8faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6d7200640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 0x7fe6bc0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005fd0 tx=0x7fe6d0009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6d8faa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802a9f0 tx=0x7fe6c8005de0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 -- 172.21.15.111:0/1910411784 shutdown_connections 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6d8faa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 --2- 172.21.15.111:0/1910411784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6d851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 -- 172.21.15.111:0/1910411784 >> 172.21.15.111:0/1910411784 conn(0x55fdb6d81310 msgr2=0x55fdb6d816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 -- 172.21.15.111:0/1910411784 shutdown_connections 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.151+0000 7fe6e0247740 1 -- 172.21.15.111:0/1910411784 wait complete. 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.152+0000 7fe6e0247740 1 -- start start 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.152+0000 7fe6e0247740 1 Processor -- start 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.152+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.152+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 0x55fdb6d8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6e0247740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6d92bb0 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6e0247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6ca4990 con 0x55fdb6d84e00 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6e0247740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fdb6d92d30 con 0x55fdb6d85720 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6e0247740 10 client.0 _ll_register_callbacks cb 0x55fdb6ca6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:56248/0 (socket says 172.21.15.111:56248) 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 -- 172.21.15.111:0/2477468941 learned_addr learned my addr 172.21.15.111:0/2477468941 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 0x55fdb6d8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 -- 172.21.15.111:0/2477468941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 msgr2=0x55fdb6d8d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 0x55fdb6d8d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 -- 172.21.15.111:0/2477468941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 msgr2=0x55fdb6d8daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6d6800640 1 -- 172.21.15.111:0/2477468941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb6ac9620 con 0x55fdb6d84e00 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.153+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 0x55fdb6d8d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80048f0 tx=0x7fe6c8004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e99185cafba0984 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8052060 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fe6c8004200 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c8004400 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6c8035070 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6e0247740 10 client.25496 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fdb6d9eb40 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.154+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004400 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe6c80317e0 con 0x55fdb6d84e00 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 10 client.25496 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 20 client.25496 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6d9fe90 0x55fdb6da2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 4 client.25496 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb6cf5ff0 con 0x55fdb6d9fe90 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.155+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6d9fe90 0x55fdb6da2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.156+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6d9fe90 0x55fdb6da2280 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be28a807c5bf7a8f server_cookie=83152d04ba2aa2cd in_seq=0 out_seq=0 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.156+0000 7fe6bbe00640 10 client.25496 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2477468941 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdb6cf5ff0 con 0x55fdb6d9fe90 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6bbe00640 10 client.25496 handle_command_reply: tid=0 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 client.25496 shutdown 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 msgr2=0x55fdb6d8d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80048f0 tx=0x7fe6c8004920 comp rx=0 tx=0).stop 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 shutdown_connections 2024-04-24T22:44:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6d9fe90 0x55fdb6da2280 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d85720 0x55fdb6d8d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6bbe00640 0 client.25496 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2477468941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d84e00 0x55fdb6d8d020 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 >> 172.21.15.111:0/2477468941 conn(0x55fdb6d81310 msgr2=0x55fdb6d87040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 shutdown_connections 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 1 -- 172.21.15.111:0/2477468941 wait complete. 2024-04-24T22:44:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.157+0000 7fe6e0247740 20 client.25496 trim_cache size 0 max 16384 2024-04-24T22:44:38.164 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:37 smithi171 ceph-mon[32337]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.173+0000 7fe6e0247740 1 Processor -- start 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 -- start start 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b61e0 con 0x55fdb6d85720 2024-04-24T22:44:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b3ab0 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6e0247740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b1920 con 0x55fdb6d84e00 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47516/0 (socket says 172.21.15.111:47516) 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6d6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52058/0 (socket says 172.21.15.111:52058) 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6cfe00640 1 -- 172.21.15.111:0/2272924474 learned_addr learned my addr 172.21.15.111:0/2272924474 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.174+0000 7fe6d7200640 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 -- 172.21.15.111:0/2272924474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d8ffe0 msgr2=0x55fdb6d8d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 -- 172.21.15.111:0/2272924474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 msgr2=0x55fdb6da57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 -- 172.21.15.111:0/2272924474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb6d923d0 con 0x55fdb6d84e00 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8031550 tx=0x7fe6c802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=514c7e6824cd1c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6c6a00640 1 -- 172.21.15.111:0/2272924474 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8052030 con 0x55fdb6d84e00 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6c6a00640 1 -- 172.21.15.111:0/2272924474 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004990 con 0x55fdb6d84e00 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.175+0000 7fe6c6a00640 1 -- 172.21.15.111:0/2272924474 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8003b80 con 0x55fdb6d84e00 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 -- 172.21.15.111:0/2272924474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 msgr2=0x55fdb6da52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8031550 tx=0x7fe6c802a400 comp rx=0 tx=0).stop 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 -- 172.21.15.111:0/2272924474 shutdown_connections 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6d8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d85720 0x55fdb6da57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2272924474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d84e00 0x55fdb6da52a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 -- 172.21.15.111:0/2272924474 >> 172.21.15.111:0/2272924474 conn(0x55fdb6d81310 msgr2=0x55fdb6d87040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 -- 172.21.15.111:0/2272924474 shutdown_connections 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.176+0000 7fe6e0247740 1 -- 172.21.15.111:0/2272924474 wait complete. 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 -- start start 2024-04-24T22:44:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 Processor -- start 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d84e00 0x55fdb6da2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6da3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b1f70 con 0x55fdb6d84e00 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b33e0 con 0x55fdb6d85720 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.177+0000 7fe6e0247740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b2640 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6e0247740 10 client.0 _ll_register_callbacks cb 0x7fe6c00d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52074/0 (socket says 172.21.15.111:52074) 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 -- 172.21.15.111:0/2145439762 learned_addr learned my addr 172.21.15.111:0/2145439762 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6cfe00640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6da3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d84e00 0x55fdb6da2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 -- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 msgr2=0x55fdb6da3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6da3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 -- 172.21.15.111:0/2145439762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d84e00 msgr2=0x55fdb6da2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d84e00 0x55fdb6da2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 -- 172.21.15.111:0/2145439762 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb6e05070 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6d7200640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00114f0 tx=0x7fe6d0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a2bcb4d19b8431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.178+0000 7fe6bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004b20 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c8004990 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fe6d0004cc0 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6d000f810 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 10 client.25516 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fdb6e06360 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004990 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe6d0005620 con 0x55fdb6d8ffe0 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 10 client.25516 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 20 client.25516 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.179+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6e06b60 0x55fdb6e08f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.180+0000 7fe6e0247740 4 client.25516 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.180+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdb6cf0340 con 0x55fdb6e06b60 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.180+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6e06b60 0x55fdb6e08f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.180+0000 7fe6d6800640 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6e06b60 0x55fdb6e08f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8e93120a79be51 server_cookie=eaf886d098d76ced in_seq=0 out_seq=0 2024-04-24T22:44:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.180+0000 7fe6bbe00640 10 client.25516 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6bbe00640 1 -- 172.21.15.111:0/2145439762 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fdb6cf0340 con 0x55fdb6e06b60 2024-04-24T22:44:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6bbe00640 10 client.25516 handle_command_reply: tid=0 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 client.25516 shutdown 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 msgr2=0x55fdb6da36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00114f0 tx=0x7fe6d0004a40 comp rx=0 tx=0).stop 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 shutdown_connections 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fdb6e06b60 0x55fdb6e08f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdb6d8ffe0 0x55fdb6da36a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fdb6d85720 0x55fdb6da3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 --2- 172.21.15.111:0/2145439762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdb6d84e00 0x55fdb6da2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6bbe00640 0 client.25516 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 >> 172.21.15.111:0/2145439762 conn(0x55fdb6d81310 msgr2=0x55fdb6d7fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 shutdown_connections 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.194+0000 7fe6e0247740 1 -- 172.21.15.111:0/2145439762 wait complete. 2024-04-24T22:44:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6e0247740 20 client.25516 trim_cache size 0 max 16384 2024-04-24T22:44:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:44:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:44:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:44:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 msgr2=0x7fe6bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 0x7fe6bc0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005fd0 tx=0x7fe6d0009500 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 msgr2=0x7fe6d819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e9f0 tx=0x7fe6c000eeb0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 shutdown_connections 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6d81a2e60 0x7fe6d81a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6d810d070 0x7fe6d819eb30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe6bc0a2e20 0x7fe6bc0a52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 --2- 172.21.15.111:0/1958206987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8107fa0 0x7fe6d819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.195+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 >> 172.21.15.111:0/1958206987 conn(0x7fe6d8075cd0 msgr2=0x7fe6d8077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.196+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 shutdown_connections 2024-04-24T22:44:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:38.196+0000 7fe6dd400640 1 -- 172.21.15.111:0/1958206987 wait complete. 2024-04-24T22:44:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:37 smithi186 ceph-mon[32753]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:37 smithi111 ceph-mon[27746]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-24T22:44:40.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:39 smithi171 ceph-mon[32337]: pgmap v84: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-24T22:44:40.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:39 smithi186 ceph-mon[32753]: pgmap v84: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-24T22:44:40.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:39 smithi111 ceph-mon[27746]: pgmap v84: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-24T22:44:42.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:41 smithi171 ceph-mon[32337]: pgmap v85: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-24T22:44:42.291 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:41 smithi111 ceph-mon[27746]: pgmap v85: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-24T22:44:42.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:41 smithi186 ceph-mon[32753]: pgmap v85: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-24T22:44:44.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:43 smithi171 ceph-mon[32337]: pgmap v86: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:44.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:43 smithi186 ceph-mon[32753]: pgmap v86: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:44.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:43 smithi111 ceph-mon[27746]: pgmap v86: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:46.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:45 smithi171 ceph-mon[32337]: pgmap v87: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:45 smithi186 ceph-mon[32753]: pgmap v87: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:46.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:45 smithi111 ceph-mon[27746]: pgmap v87: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:48.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:47 smithi171 ceph-mon[32337]: pgmap v88: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:48.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:47 smithi186 ceph-mon[32753]: pgmap v88: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:47 smithi111 ceph-mon[27746]: pgmap v88: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-24T22:44:50.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:49 smithi171 ceph-mon[32337]: pgmap v89: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:50.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:49 smithi186 ceph-mon[32753]: pgmap v89: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:50.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:49 smithi111 ceph-mon[27746]: pgmap v89: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:44:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 -- 172.21.15.111:0/4068986553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 msgr2=0x7ff6501149d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 --2- 172.21.15.111:0/4068986553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501149d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff648009ca0 tx=0x7ff64802a8c0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 -- 172.21.15.111:0/4068986553 shutdown_connections 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 --2- 172.21.15.111:0/4068986553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501149d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 --2- 172.21.15.111:0/4068986553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650076590 0x7ff6501120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 --2- 172.21.15.111:0/4068986553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650075c70 0x7ff650076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 -- 172.21.15.111:0/4068986553 >> 172.21.15.111:0/4068986553 conn(0x7ff6500ff470 msgr2=0x7ff650101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 -- 172.21.15.111:0/4068986553 shutdown_connections 2024-04-24T22:44:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.336+0000 7ff655800640 1 -- 172.21.15.111:0/4068986553 wait complete. 2024-04-24T22:44:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.337+0000 7ff655800640 1 Processor -- start 2024-04-24T22:44:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.337+0000 7ff655800640 1 -- start start 2024-04-24T22:44:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650075c70 0x7ff65019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650076590 0x7ff65019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff650118180 con 0x7ff6501125e0 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff650118000 con 0x7ff650076590 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff655800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff650118300 con 0x7ff650075c70 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff64f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff64f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34844/0 (socket says 172.21.15.111:34844) 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff64f400640 1 -- 172.21.15.111:0/3883398054 learned_addr learned my addr 172.21.15.111:0/3883398054 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff64ea00640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650075c70 0x7ff65019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.338+0000 7ff646000640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650076590 0x7ff65019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 -- 172.21.15.111:0/3883398054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650075c70 msgr2=0x7ff65019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650075c70 0x7ff65019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 -- 172.21.15.111:0/3883398054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650076590 msgr2=0x7ff65019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650076590 0x7ff65019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 -- 172.21.15.111:0/3883398054 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6501a3520 con 0x7ff6501125e0 2024-04-24T22:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff64f400640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff648003f40 tx=0x7ff648003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff647e00640 1 -- 172.21.15.111:0/3883398054 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480317d0 con 0x7ff6501125e0 2024-04-24T22:44:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff647e00640 1 -- 172.21.15.111:0/3883398054 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff648031970 con 0x7ff6501125e0 2024-04-24T22:44:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff647e00640 1 -- 172.21.15.111:0/3883398054 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648030c90 con 0x7ff6501125e0 2024-04-24T22:44:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.339+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6501a37b0 con 0x7ff6501125e0 2024-04-24T22:44:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.340+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6501ab0a0 con 0x7ff6501125e0 2024-04-24T22:44:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff647e00640 1 -- 172.21.15.111:0/3883398054 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff648004040 con 0x7ff6501125e0 2024-04-24T22:44:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 Processor -- start 2024-04-24T22:44:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 -- start start 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24181720 0x557f2418baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f2418bfe0 0x557f2418e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f240e4ab0 con 0x557f24181720 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f2408d640 con 0x557f24180e00 2024-04-24T22:44:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.343+0000 7ff658613740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f240e0a70 con 0x557f2418bfe0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36618/0 (socket says 172.21.15.111:36618) 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 -- 172.21.15.111:0/1129075655 learned_addr learned my addr 172.21.15.111:0/1129075655 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f2418bfe0 0x557f2418e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff64ea00640 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24181720 0x557f2418baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff647e00640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 0x7ff6340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 -- 172.21.15.111:0/1129075655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f2418bfe0 msgr2=0x557f2418e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f2418bfe0 0x557f2418e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 -- 172.21.15.111:0/1129075655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24181720 msgr2=0x557f2418baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24181720 0x557f2418baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff646000640 1 -- 172.21.15.111:0/1129075655 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f23df9210 con 0x557f24180e00 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.344+0000 7ff647e00640 1 -- 172.21.15.111:0/3883398054 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff6480f0980 con 0x7ff6501125e0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff64f400640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 0x7ff6340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff646000640 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x557f2417e4a0 tx=0x7ff638009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ad147eed86a68f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff63fe00640 1 -- 172.21.15.111:0/1129075655 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638015730 con 0x557f24180e00 2024-04-24T22:44:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff63fe00640 1 -- 172.21.15.111:0/1129075655 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff63801d070 con 0x557f24180e00 2024-04-24T22:44:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff64f400640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 0x7ff6340a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff648006010 tx=0x7ff6480095c0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.345+0000 7ff63fe00640 1 -- 172.21.15.111:0/1129075655 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638018ac0 con 0x557f24180e00 2024-04-24T22:44:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 -- 172.21.15.111:0/1129075655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 msgr2=0x557f241811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x557f2417e4a0 tx=0x7ff638009510 comp rx=0 tx=0).stop 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 -- 172.21.15.111:0/1129075655 shutdown_connections 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f2418bfe0 0x557f2418e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24181720 0x557f2418baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 --2- 172.21.15.111:0/1129075655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f241811e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 -- 172.21.15.111:0/1129075655 >> 172.21.15.111:0/1129075655 conn(0x557f2417d310 msgr2=0x557f2417d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 -- 172.21.15.111:0/1129075655 shutdown_connections 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.346+0000 7ff658613740 1 -- 172.21.15.111:0/1129075655 wait complete. 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 -- start start 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 Processor -- start 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24180e00 0x557f24188fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 0x557f24189a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f2418ebb0 con 0x557f24180e00 2024-04-24T22:44:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f240a0990 con 0x557f2418bfe0 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f2418ed30 con 0x557f24181720 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.347+0000 7ff658613740 10 client.0 _ll_register_callbacks cb 0x557f240a2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff646000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24180e00 0x557f24188fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52060/0 (socket says 172.21.15.111:52060) 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 -- 172.21.15.111:0/1326525099 learned_addr learned my addr 172.21.15.111:0/1326525099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 -- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 msgr2=0x557f24189a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff646000640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 0x557f24189a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 0x557f24189a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 -- 172.21.15.111:0/1326525099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24180e00 msgr2=0x557f24188fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24180e00 0x557f24188fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 -- 172.21.15.111:0/1326525099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f23ec5620 con 0x557f24181720 2024-04-24T22:44:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff646000640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 0x557f24189a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff64ea00640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff640010a50 tx=0x7ff640010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0290918eb872e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.348+0000 7ff63cc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64001e060 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff63801d070 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff64000fbd0 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff64000fdd0 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff658613740 10 client.25525 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557f2419abd0 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.349+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff63801d070 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff6400214c0 con 0x557f24181720 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 10 client.25525 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 20 client.25525 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f2419c040 0x557f2419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 4 client.25525 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:44:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f240f1ff0 con 0x557f2419c040 2024-04-24T22:44:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.350+0000 7ff646000640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f2419c040 0x557f2419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.352+0000 7ff646000640 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f2419c040 0x557f2419e4a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309b3f107e26616d server_cookie=debcbe4977f212b5 in_seq=0 out_seq=0 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.352+0000 7ff63cc00640 10 client.25525 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff63cc00640 1 -- 172.21.15.111:0/1326525099 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f240f1ff0 con 0x557f2419c040 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff63cc00640 10 client.25525 handle_command_reply: tid=0 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 client.25525 shutdown 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 msgr2=0x557f24189510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff640010a50 tx=0x7ff640010f10 comp rx=0 tx=0).stop 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 shutdown_connections 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f2419c040 0x557f2419e4a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f2418bfe0 0x557f24189a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24189510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 --2- 172.21.15.111:0/1326525099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f24180e00 0x557f24188fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff63cc00640 0 client.25525 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 >> 172.21.15.111:0/1326525099 conn(0x557f2417d310 msgr2=0x557f24183090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 shutdown_connections 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 1 -- 172.21.15.111:0/1326525099 wait complete. 2024-04-24T22:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.353+0000 7ff658613740 20 client.25525 trim_cache size 0 max 16384 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 Processor -- start 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 -- start start 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f2417cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24182ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f241a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b6180 con 0x557f2418bfe0 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b3a50 con 0x557f24180e00 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff658613740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b18a0 con 0x557f24181720 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff646000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f2417cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff646000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f2417cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36644/0 (socket says 172.21.15.111:36644) 2024-04-24T22:44:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff64ea00640 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24182ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff646000640 1 -- 172.21.15.111:0/62804126 learned_addr learned my addr 172.21.15.111:0/62804126 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.370+0000 7ff646000640 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f241a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 -- 172.21.15.111:0/62804126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 msgr2=0x557f2417cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f2417cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 -- 172.21.15.111:0/62804126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 msgr2=0x557f241a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f241a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 -- 172.21.15.111:0/62804126 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2417d140 con 0x557f24181720 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff64ea00640 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24182ec0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff640021860 tx=0x7ff64000ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1239a726c7cb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.371+0000 7ff63f400640 1 -- 172.21.15.111:0/62804126 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64000a340 con 0x557f24181720 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff63f400640 1 -- 172.21.15.111:0/62804126 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64000fbd0 con 0x557f24181720 2024-04-24T22:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff63f400640 1 -- 172.21.15.111:0/62804126 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640023bc0 con 0x557f24181720 2024-04-24T22:44:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 -- 172.21.15.111:0/62804126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 msgr2=0x557f24182ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24182ec0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff640021860 tx=0x7ff64000ffd0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 -- 172.21.15.111:0/62804126 shutdown_connections 2024-04-24T22:44:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f241a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24181720 0x557f24182ec0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 --2- 172.21.15.111:0/62804126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24180e00 0x557f2417cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 -- 172.21.15.111:0/62804126 >> 172.21.15.111:0/62804126 conn(0x557f2417d310 msgr2=0x557f241827b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 -- 172.21.15.111:0/62804126 shutdown_connections 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.372+0000 7ff658613740 1 -- 172.21.15.111:0/62804126 wait complete. 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.373+0000 7ff658613740 1 -- start start 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.373+0000 7ff658613740 1 Processor -- start 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.373+0000 7ff658613740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.373+0000 7ff658613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24181720 0x557f24190f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.373+0000 7ff658613740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f24191490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff658613740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b1f10 con 0x557f2418bfe0 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff658613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b3380 con 0x557f24181720 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff658613740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b25e0 con 0x557f24180e00 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff658613740 10 client.0 _ll_register_callbacks cb 0x7ff64000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52084/0 (socket says 172.21.15.111:52084) 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 -- 172.21.15.111:0/752041955 learned_addr learned my addr 172.21.15.111:0/752041955 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff64ea00640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24181720 0x557f24190f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f24191490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 -- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24181720 msgr2=0x557f24190f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24181720 0x557f24190f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 -- 172.21.15.111:0/752041955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 msgr2=0x557f24191490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f24191490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 -- 172.21.15.111:0/752041955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f241919d0 con 0x557f24180e00 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff646000640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff638017730 tx=0x7ff63802b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8d70459f57bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.374+0000 7ff63cc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638018d50 con 0x557f24180e00 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7ff638015730 con 0x557f24180e00 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64000fbd0 con 0x557f24180e00 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff638039050 con 0x557f24180e00 2024-04-24T22:44:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff658613740 10 client.25531 fetch_fsmap learned FSMap version 24 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557f24203320 con 0x557f24180e00 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64000fbd0 con 0x557f24180e00 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.375+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff6380237e0 con 0x557f24180e00 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 10 client.25531 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 20 client.25531 populate_metadata read hostname 'smithi111' 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f242036c0 0x557f24205ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 4 client.25531 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f240ec340 con 0x557f242036c0 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.376+0000 7ff64ea00640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f242036c0 0x557f24205ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.377+0000 7ff64ea00640 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f242036c0 0x557f24205ab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4ef61accc45910 server_cookie=4044bbe3c00502c8 in_seq=0 out_seq=0 2024-04-24T22:44:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.377+0000 7ff63cc00640 10 client.25531 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.390+0000 7ff63cc00640 1 -- 172.21.15.111:0/752041955 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f240ec340 con 0x557f242036c0 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.390+0000 7ff63cc00640 10 client.25531 handle_command_reply: tid=0 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.390+0000 7ff658613740 1 client.25531 shutdown 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 msgr2=0x557f24194170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff638017730 tx=0x7ff63802b040 comp rx=0 tx=0).stop 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 shutdown_connections 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557f242036c0 0x557f24205ab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f2418bfe0 0x557f24191490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24181720 0x557f24190f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff63cc00640 0 client.25531 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 --2- 172.21.15.111:0/752041955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f24180e00 0x557f24194170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 >> 172.21.15.111:0/752041955 conn(0x557f2417d310 msgr2=0x557f24183040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 shutdown_connections 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 1 -- 172.21.15.111:0/752041955 wait complete. 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.391+0000 7ff658613740 20 client.25531 trim_cache size 0 max 16384 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:44:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 msgr2=0x7ff6340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 0x7ff6340a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff648006010 tx=0x7ff6480095c0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 msgr2=0x7ff6501a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff648003f40 tx=0x7ff648003f70 comp rx=0 tx=0).stop 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 shutdown_connections 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6501125e0 0x7ff6501a2df0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff650076590 0x7ff65019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff6340a2da0 0x7ff6340a5260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 --2- 172.21.15.111:0/3883398054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff650075c70 0x7ff65019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 >> 172.21.15.111:0/3883398054 conn(0x7ff6500ff470 msgr2=0x7ff650101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 shutdown_connections 2024-04-24T22:44:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:44:51.392+0000 7ff655800640 1 -- 172.21.15.111:0/3883398054 wait complete. 2024-04-24T22:44:52.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:51 smithi171 ceph-mon[32337]: pgmap v90: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:52.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:51 smithi186 ceph-mon[32753]: pgmap v90: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:52.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:51 smithi111 ceph-mon[27746]: pgmap v90: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:54.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:53 smithi171 ceph-mon[32337]: pgmap v91: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-24T22:44:54.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:53 smithi186 ceph-mon[32753]: pgmap v91: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-24T22:44:54.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:53 smithi111 ceph-mon[27746]: pgmap v91: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-24T22:44:56.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:55 smithi171 ceph-mon[32337]: pgmap v92: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:56.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:55 smithi186 ceph-mon[32753]: pgmap v92: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:56.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:55 smithi111 ceph-mon[27746]: pgmap v92: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:58.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:57 smithi171 ceph-mon[32337]: pgmap v93: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:58.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:57 smithi186 ceph-mon[32753]: pgmap v93: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:44:58.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:57 smithi111 ceph-mon[27746]: pgmap v93: 97 pgs: 97 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-24T22:45:00.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:44:59 smithi171 ceph-mon[32337]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:45:00.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:44:59 smithi186 ceph-mon[32753]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:45:00.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:44:59 smithi111 ceph-mon[27746]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-24T22:45:02.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:01 smithi171 ceph-mon[32337]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-24T22:45:02.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:01 smithi186 ceph-mon[32753]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-24T22:45:02.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:01 smithi111 ceph-mon[27746]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-24T22:45:04.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:03 smithi171 ceph-mon[32337]: pgmap v96: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-04-24T22:45:04.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:03 smithi186 ceph-mon[32753]: pgmap v96: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-04-24T22:45:04.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:03 smithi111 ceph-mon[27746]: pgmap v96: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 -- 172.21.15.111:0/619258945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90108950 msgr2=0x7f6c9010c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 --2- 172.21.15.111:0/619258945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90108950 0x7f6c9010c990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009860 tx=0x7f6c7c02a340 comp rx=0 tx=0).stop 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 -- 172.21.15.111:0/619258945 shutdown_connections 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 --2- 172.21.15.111:0/619258945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c9010d070 0x7f6c90114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 --2- 172.21.15.111:0/619258945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90108950 0x7f6c9010c990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 --2- 172.21.15.111:0/619258945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c90107fa0 0x7f6c90108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 -- 172.21.15.111:0/619258945 >> 172.21.15.111:0/619258945 conn(0x7f6c90075cd0 msgr2=0x7f6c900780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 -- 172.21.15.111:0/619258945 shutdown_connections 2024-04-24T22:45:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.534+0000 7f6c98200640 1 -- 172.21.15.111:0/619258945 wait complete. 2024-04-24T22:45:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.535+0000 7f6c98200640 1 Processor -- start 2024-04-24T22:45:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.535+0000 7f6c98200640 1 -- start start 2024-04-24T22:45:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c90107fa0 0x7f6c9019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c9010d070 0x7f6c9019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901180e0 con 0x7f6c901a2dd0 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90117f60 con 0x7f6c9010d070 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c98200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90118260 con 0x7f6c90107fa0 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c96200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c96200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47698/0 (socket says 172.21.15.111:47698) 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.536+0000 7f6c96200640 1 -- 172.21.15.111:0/2416454746 learned_addr learned my addr 172.21.15.111:0/2416454746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 -- 172.21.15.111:0/2416454746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c90107fa0 msgr2=0x7f6c9019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c90107fa0 0x7f6c9019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 -- 172.21.15.111:0/2416454746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c9010d070 msgr2=0x7f6c9019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c9010d070 0x7f6c9019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 -- 172.21.15.111:0/2416454746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c901a38c0 con 0x7f6c901a2dd0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c96200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800efc0 tx=0x7f6c8800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c8e000640 1 -- 172.21.15.111:0/2416454746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88003970 con 0x7f6c901a2dd0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c8e000640 1 -- 172.21.15.111:0/2416454746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c88004020 con 0x7f6c901a2dd0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c901a3bb0 con 0x7f6c901a2dd0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c8e000640 1 -- 172.21.15.111:0/2416454746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800b9f0 con 0x7f6c901a2dd0 2024-04-24T22:45:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.537+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c901ab350 con 0x7f6c901a2dd0 2024-04-24T22:45:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 Processor -- start 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c8e000640 1 -- 172.21.15.111:0/2416454746 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6c8801d020 con 0x7f6c901a2dd0 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 -- start start 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970da53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b970cfbab0 con 0x55b970da2fe0 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b970ca4640 con 0x55b970d97e00 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.541+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b970cf7a70 con 0x55b970d98720 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:42060/0 (socket says 172.21.15.111:42060) 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 -- 172.21.15.111:0/1207819859 learned_addr learned my addr 172.21.15.111:0/1207819859 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970da53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 -- 172.21.15.111:0/1207819859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 msgr2=0x55b970da2aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da2aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 -- 172.21.15.111:0/1207819859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 msgr2=0x55b970da53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c8e000640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 0x7f6c780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970da53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 -- 172.21.15.111:0/1207819859 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b970a10210 con 0x55b970d97e00 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c94e00640 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005e00 tx=0x7f6c7c005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1001a9f07a718317 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c8e000640 1 -- 172.21.15.111:0/2416454746 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f6c880cf3e0 con 0x7f6c901a2dd0 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.542+0000 7f6c96200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 0x7f6c780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c82a00640 1 -- 172.21.15.111:0/1207819859 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c037040 con 0x55b970d97e00 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c82a00640 1 -- 172.21.15.111:0/1207819859 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c032440 con 0x55b970d97e00 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c82a00640 1 -- 172.21.15.111:0/1207819859 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0405f0 con 0x55b970d97e00 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c9afc5740 1 -- 172.21.15.111:0/1207819859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 msgr2=0x55b970d981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c96200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 0x7f6c780a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ccb60 tx=0x7f6c8800ceb0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.543+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005e00 tx=0x7f6c7c005af0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 -- 172.21.15.111:0/1207819859 shutdown_connections 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970da53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/1207819859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d97e00 0x55b970d981e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 -- 172.21.15.111:0/1207819859 >> 172.21.15.111:0/1207819859 conn(0x55b970d94310 msgr2=0x55b970d946f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 -- 172.21.15.111:0/1207819859 shutdown_connections 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 -- 172.21.15.111:0/1207819859 wait complete. 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 -- start start 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.544+0000 7f6c9afc5740 1 Processor -- start 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970da0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b970da5bb0 con 0x55b970d97e00 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b970cb7990 con 0x55b970da2fe0 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b970da5d30 con 0x55b970d98720 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c9afc5740 10 client.0 _ll_register_callbacks cb 0x55b970cb9950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47728/0 (socket says 172.21.15.111:47728) 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c94e00640 1 -- 172.21.15.111:0/3009904619 learned_addr learned my addr 172.21.15.111:0/3009904619 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c95800640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.545+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970da0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 -- 172.21.15.111:0/3009904619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 msgr2=0x55b970da0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 -- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 msgr2=0x55b970da0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970da0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 -- 172.21.15.111:0/3009904619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b970adc620 con 0x55b970d97e00 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c030460 tx=0x7f6c7c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872b36e02f94d2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c77400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c051060 con 0x55b970d97e00 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f6c7c0325f0 con 0x55b970d97e00 2024-04-24T22:45:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.546+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c032440 con 0x55b970d97e00 2024-04-24T22:45:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c7c037040 con 0x55b970d97e00 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b970db1b40 con 0x55b970d97e00 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c032440 con 0x55b970d97e00 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6c7c03d070 con 0x55b970d97e00 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 20 client.15447 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970db2e90 0x55b970db5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 4 client.15447 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.547+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b970d08ff0 con 0x55b970db2e90 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.548+0000 7f6c95800640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970db2e90 0x55b970db5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.548+0000 7f6c95800640 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970db2e90 0x55b970db5280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0a74b3f4c14a3a server_cookie=3a1fbd828b274e24 in_seq=0 out_seq=0 2024-04-24T22:45:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.548+0000 7f6c77400640 10 client.15447 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.549+0000 7f6c77400640 1 -- 172.21.15.111:0/3009904619 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b970d08ff0 con 0x55b970db2e90 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.549+0000 7f6c77400640 10 client.15447 handle_command_reply: tid=0 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.549+0000 7f6c9afc5740 1 client.15447 shutdown 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 msgr2=0x55b970da0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c030460 tx=0x7f6c7c032dd0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 shutdown_connections 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970db2e90 0x55b970db5280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970da0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d98720 0x55b970da0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3009904619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d97e00 0x55b970da0030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 >> 172.21.15.111:0/3009904619 conn(0x55b970d94310 msgr2=0x55b970d9a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c77400640 0 client.15447 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 shutdown_connections 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3009904619 wait complete. 2024-04-24T22:45:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.550+0000 7f6c9afc5740 20 client.15447 trim_cache size 0 max 16384 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 Processor -- start 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 -- start start 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970daea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970d93300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b6060 con 0x55b970da2fe0 2024-04-24T22:45:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b3930 con 0x55b970d98720 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.566+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b17a0 con 0x55b970d97e00 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970daea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:42082/0 (socket says 172.21.15.111:42082) 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 -- 172.21.15.111:0/319970262 learned_addr learned my addr 172.21.15.111:0/319970262 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c94e00640 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970d93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 -- 172.21.15.111:0/319970262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 msgr2=0x55b970daea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970daea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 -- 172.21.15.111:0/319970262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 msgr2=0x55b970d93300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970d93300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c95800640 1 -- 172.21.15.111:0/319970262 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b970da5ca0 con 0x55b970d98720 2024-04-24T22:45:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.567+0000 7f6c94e00640 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970d93300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.568+0000 7f6c95800640 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55b970d9f890 tx=0x7f6c84005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f814cca05abd1880 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.568+0000 7f6c82000640 1 -- 172.21.15.111:0/319970262 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8401abd0 con 0x55b970d98720 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.568+0000 7f6c82000640 1 -- 172.21.15.111:0/319970262 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8401a870 con 0x55b970d98720 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.568+0000 7f6c82000640 1 -- 172.21.15.111:0/319970262 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c840047a0 con 0x55b970d98720 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- 172.21.15.111:0/319970262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 msgr2=0x55b970da08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55b970d9f890 tx=0x7f6c84005f70 comp rx=0 tx=0).stop 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- 172.21.15.111:0/319970262 shutdown_connections 2024-04-24T22:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970da2fe0 0x55b970d93300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970d98720 0x55b970da08d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/319970262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970daea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- 172.21.15.111:0/319970262 >> 172.21.15.111:0/319970262 conn(0x55b970d94310 msgr2=0x55b970d92bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- 172.21.15.111:0/319970262 shutdown_connections 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- 172.21.15.111:0/319970262 wait complete. 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 -- start start 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.569+0000 7f6c9afc5740 1 Processor -- start 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970d9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970e180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b1df0 con 0x55b970d98720 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b3260 con 0x55b970da2fe0 2024-04-24T22:45:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b24c0 con 0x55b970d97e00 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c9afc5740 10 client.0 _ll_register_callbacks cb 0x7f6c88009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970e180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47738/0 (socket says 172.21.15.111:47738) 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 -- 172.21.15.111:0/3531586754 learned_addr learned my addr 172.21.15.111:0/3531586754 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970d9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 -- 172.21.15.111:0/3531586754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 msgr2=0x55b970d9d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970d9d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 -- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 msgr2=0x55b970e180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970e180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.570+0000 7f6c95800640 1 -- 172.21.15.111:0/3531586754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b970e18600 con 0x55b970d98720 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c95800640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000c00 tx=0x7f6c8400a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bb136fd108b0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c77400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84028030 con 0x55b970d98720 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f6c8401f070 con 0x55b970d98720 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8401a870 con 0x55b970d98720 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c84005230 con 0x55b970d98720 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c9afc5740 10 client.15453 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.571+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b970e197b0 con 0x55b970d98720 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8401a870 con 0x55b970d98720 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6c8401a070 con 0x55b970d98720 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 20 client.15453 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970e19e70 0x55b970e1c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 4 client.15453 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b970d03340 con 0x55b970e19e70 2024-04-24T22:45:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.572+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970e19e70 0x55b970e1c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.573+0000 7f6c77400640 10 client.15453 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.573+0000 7f6c94e00640 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970e19e70 0x55b970e1c290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97595da67ac6c04 server_cookie=4d27b4761ed3e1d9 in_seq=0 out_seq=0 2024-04-24T22:45:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c77400640 1 -- 172.21.15.111:0/3531586754 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b970d03340 con 0x55b970e19e70 2024-04-24T22:45:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c77400640 10 client.15453 handle_command_reply: tid=0 2024-04-24T22:45:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 client.15453 shutdown 2024-04-24T22:45:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 msgr2=0x55b970d9d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000c00 tx=0x7f6c8400a830 comp rx=0 tx=0).stop 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 shutdown_connections 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55b970e19e70 0x55b970e1c290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b970da2fe0 0x55b970e180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b970d98720 0x55b970d9d8e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c77400640 0 client.15453 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.586+0000 7f6c9afc5740 1 --2- 172.21.15.111:0/3531586754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b970d97e00 0x55b970d9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.587+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 >> 172.21.15.111:0/3531586754 conn(0x55b970d94310 msgr2=0x55b970d9a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.587+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 shutdown_connections 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.587+0000 7f6c9afc5740 1 -- 172.21.15.111:0/3531586754 wait complete. 2024-04-24T22:45:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.587+0000 7f6c9afc5740 20 client.15453 trim_cache size 0 max 16384 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 msgr2=0x7f6c780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 0x7f6c780a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ccb60 tx=0x7f6c8800ceb0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 msgr2=0x7f6c901a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800efc0 tx=0x7f6c8800c800 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 shutdown_connections 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901a2dd0 0x7f6c901a3280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c9010d070 0x7f6c9019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f6c780a2da0 0x7f6c780a5260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 --2- 172.21.15.111:0/2416454746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c90107fa0 0x7f6c9019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 >> 172.21.15.111:0/2416454746 conn(0x7f6c90075cd0 msgr2=0x7f6c90076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 shutdown_connections 2024-04-24T22:45:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:04.588+0000 7f6c98200640 1 -- 172.21.15.111:0/2416454746 wait complete. 2024-04-24T22:45:06.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:05 smithi171 ceph-mon[32337]: pgmap v97: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:06.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:05 smithi186 ceph-mon[32753]: pgmap v97: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:06.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:05 smithi111 ceph-mon[27746]: pgmap v97: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:08.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:07 smithi171 ceph-mon[32337]: pgmap v98: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:08.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:07 smithi186 ceph-mon[32753]: pgmap v98: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:08.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:07 smithi111 ceph-mon[27746]: pgmap v98: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T22:45:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:09 smithi171 ceph-mon[32337]: pgmap v99: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-24T22:45:10.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:09 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:10.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:09 smithi186 ceph-mon[32753]: pgmap v99: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-24T22:45:10.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:09 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:10.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:09 smithi111 ceph-mon[27746]: pgmap v99: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-24T22:45:10.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:09 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:10 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:10 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:11.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:10 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:45:11.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:10 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:11.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:10 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:11.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:10 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:45:11.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:10 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:11.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:10 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:11.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:10 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:45:12.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:11 smithi171 ceph-mon[32337]: pgmap v100: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T22:45:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:11 smithi186 ceph-mon[32753]: pgmap v100: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T22:45:12.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:11 smithi111 ceph-mon[27746]: pgmap v100: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T22:45:14.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:13 smithi171 ceph-mon[32337]: pgmap v101: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 32 op/s 2024-04-24T22:45:14.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:13 smithi186 ceph-mon[32753]: pgmap v101: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 32 op/s 2024-04-24T22:45:14.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:13 smithi111 ceph-mon[27746]: pgmap v101: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 32 op/s 2024-04-24T22:45:16.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:15 smithi171 ceph-mon[32337]: pgmap v102: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:16.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:15 smithi186 ceph-mon[32753]: pgmap v102: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:16.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:15 smithi111 ceph-mon[27746]: pgmap v102: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 -- 172.21.15.111:0/3792516083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 msgr2=0x7f8760108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 --2- 172.21.15.111:0/3792516083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f8760108380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8750009860 tx=0x7f875002a340 comp rx=0 tx=0).stop 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 -- 172.21.15.111:0/3792516083 shutdown_connections 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 --2- 172.21.15.111:0/3792516083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f8760114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 --2- 172.21.15.111:0/3792516083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8760108950 0x7f876010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 --2- 172.21.15.111:0/3792516083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f8760108380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 -- 172.21.15.111:0/3792516083 >> 172.21.15.111:0/3792516083 conn(0x7f8760075cd0 msgr2=0x7f87600780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.731+0000 7f8767400640 1 -- 172.21.15.111:0/3792516083 shutdown_connections 2024-04-24T22:45:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.732+0000 7f8767400640 1 -- 172.21.15.111:0/3792516083 wait complete. 2024-04-24T22:45:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.732+0000 7f8767400640 1 Processor -- start 2024-04-24T22:45:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 -- start start 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 0x7f87601a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8760118120 con 0x7f876010d070 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8760117fa0 con 0x7f87601a2e70 2024-04-24T22:45:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8767400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87601182a0 con 0x7f8760107fa0 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f875fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8765400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 0x7f87601a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f875fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38730/0 (socket says 172.21.15.111:38730) 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f875fe00640 1 -- 172.21.15.111:0/1102522046 learned_addr learned my addr 172.21.15.111:0/1102522046 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.733+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44076/0 (socket says 172.21.15.111:44076) 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 -- 172.21.15.111:0/1102522046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 msgr2=0x7f876019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 -- 172.21.15.111:0/1102522046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 msgr2=0x7f87601a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 0x7f87601a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 -- 172.21.15.111:0/1102522046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601a3960 con 0x7f876010d070 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f8764a00640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f8765400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 0x7f87601a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875fe00640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f874800ebc0 tx=0x7f874800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875d600640 1 -- 172.21.15.111:0/1102522046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748004d20 con 0x7f876010d070 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875d600640 1 -- 172.21.15.111:0/1102522046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8748005500 con 0x7f876010d070 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87601a3c50 con 0x7f876010d070 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.734+0000 7f875d600640 1 -- 172.21.15.111:0/1102522046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748010890 con 0x7f876010d070 2024-04-24T22:45:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.735+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601ab3f0 con 0x7f876010d070 2024-04-24T22:45:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f875d600640 1 -- 172.21.15.111:0/1102522046 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8748005020 con 0x7f876010d070 2024-04-24T22:45:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 Processor -- start 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 -- start start 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e3e00 0x5610ef9e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610ef947ab0 con 0x5610ef9eefe0 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5610ef8f0640 con 0x5610ef9e3e00 2024-04-24T22:45:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.738+0000 7f876a395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5610ef943a70 con 0x5610ef9e4720 2024-04-24T22:45:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f875fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8765400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e3e00 0x5610ef9e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44086/0 (socket says 172.21.15.111:44086) 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 -- 172.21.15.111:0/1861313828 learned_addr learned my addr 172.21.15.111:0/1861313828 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 -- 172.21.15.111:0/1861313828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e3e00 msgr2=0x5610ef9e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e3e00 0x5610ef9e41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 -- 172.21.15.111:0/1861313828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 msgr2=0x5610ef9f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.739+0000 7f8764a00640 1 -- 172.21.15.111:0/1861313828 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ef65c210 con 0x5610ef9e4720 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f875d600640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 0x7f87400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f8764a00640 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f875002a860 tx=0x7f8750009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e8f8c722a81266 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f875d600640 1 -- 172.21.15.111:0/1102522046 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f87480cfc80 con 0x7f876010d070 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f8765400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 0x7f87400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f8755c00640 1 -- 172.21.15.111:0/1861313828 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750037040 con 0x5610ef9e4720 2024-04-24T22:45:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f8755c00640 1 -- 172.21.15.111:0/1861313828 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8750004230 con 0x5610ef9e4720 2024-04-24T22:45:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.740+0000 7f8755c00640 1 -- 172.21.15.111:0/1861313828 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750003cf0 con 0x5610ef9e4720 2024-04-24T22:45:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 -- 172.21.15.111:0/1861313828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 msgr2=0x5610ef9eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f875002a860 tx=0x7f8750009510 comp rx=0 tx=0).stop 2024-04-24T22:45:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f8765400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 0x7f87400a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8758009530 tx=0x7f8758009260 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 -- 172.21.15.111:0/1861313828 shutdown_connections 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9eeaa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 --2- 172.21.15.111:0/1861313828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e3e00 0x5610ef9e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 -- 172.21.15.111:0/1861313828 >> 172.21.15.111:0/1861313828 conn(0x5610ef9e0310 msgr2=0x5610ef9e06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 -- 172.21.15.111:0/1861313828 shutdown_connections 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.741+0000 7f876a395740 1 -- 172.21.15.111:0/1861313828 wait complete. 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 -- start start 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 Processor -- start 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610ef9f1bb0 con 0x5610ef9e4720 2024-04-24T22:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5610ef903990 con 0x5610ef9eefe0 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5610ef9f1d30 con 0x5610ef9e3e00 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.742+0000 7f876a395740 10 client.0 _ll_register_callbacks cb 0x5610ef905950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f875fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f875fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38748/0 (socket says 172.21.15.111:38748) 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f875fe00640 1 -- 172.21.15.111:0/286766707 learned_addr learned my addr 172.21.15.111:0/286766707 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8765400640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 -- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 msgr2=0x5610ef9eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 -- 172.21.15.111:0/286766707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 msgr2=0x5610ef9ec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 -- 172.21.15.111:0/286766707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ef728620 con 0x5610ef9e3e00 2024-04-24T22:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f875fe00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f8764a00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9ebfd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8750005940 tx=0x7f875002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6ae528b42b9f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.743+0000 7f874e000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750037040 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f8750030960 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8750004230 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8750003a30 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f876a395740 10 client.25552 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.744+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5610ef9f1bb0 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8750004230 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f875004eb40 con 0x5610ef9e3e00 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 10 client.25552 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 10 client.25552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 20 client.25552 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610ef9fec60 0x5610efa01050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 4 client.25552 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ef954ff0 con 0x5610ef9fec60 2024-04-24T22:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.745+0000 7f875fe00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610ef9fec60 0x5610efa01050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.746+0000 7f875fe00640 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610ef9fec60 0x5610efa01050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbb59cd571d743c server_cookie=ffd1806227455c9 in_seq=0 out_seq=0 2024-04-24T22:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.746+0000 7f874e000640 10 client.25552 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f874e000640 1 -- 172.21.15.111:0/286766707 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610ef954ff0 con 0x5610ef9fec60 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f874e000640 10 client.25552 handle_command_reply: tid=0 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 client.25552 shutdown 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 msgr2=0x5610ef9ebfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9ebfd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8750005940 tx=0x7f875002a340 comp rx=0 tx=0).stop 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 shutdown_connections 2024-04-24T22:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610ef9fec60 0x5610efa01050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e4720 0x5610ef9ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f874e000640 0 client.25552 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 --2- 172.21.15.111:0/286766707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9ebfd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 >> 172.21.15.111:0/286766707 conn(0x5610ef9e0310 msgr2=0x5610ef9e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 shutdown_connections 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 1 -- 172.21.15.111:0/286766707 wait complete. 2024-04-24T22:45:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.747+0000 7f876a395740 20 client.25552 trim_cache size 0 max 16384 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 Processor -- start 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 -- start start 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e4720 0x5610ef9e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b6190 con 0x5610ef9eefe0 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b3a60 con 0x5610ef9e4720 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.765+0000 7f876a395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b18d0 con 0x5610ef9e3e00 2024-04-24T22:45:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8764a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44100/0 (socket says 172.21.15.111:44100) 2024-04-24T22:45:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8764a00640 1 -- 172.21.15.111:0/2608493633 learned_addr learned my addr 172.21.15.111:0/2608493633 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f875fe00640 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e4720 0x5610ef9e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 -- 172.21.15.111:0/2608493633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 msgr2=0x5610ef9faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 -- 172.21.15.111:0/2608493633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e4720 msgr2=0x5610ef9e06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e4720 0x5610ef9e06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 -- 172.21.15.111:0/2608493633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ef9eaf80 con 0x5610ef9eefe0 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8764a00640 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.766+0000 7f8765400640 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9ec870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8758012010 tx=0x7f8758016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2ed0cc30641fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f8755200640 1 -- 172.21.15.111:0/2608493633 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875802c070 con 0x5610ef9eefe0 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f8755200640 1 -- 172.21.15.111:0/2608493633 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875800f4e0 con 0x5610ef9eefe0 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f8755200640 1 -- 172.21.15.111:0/2608493633 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580044e0 con 0x5610ef9eefe0 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 -- 172.21.15.111:0/2608493633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 msgr2=0x5610ef9ec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9ec870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8758012010 tx=0x7f8758016000 comp rx=0 tx=0).stop 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 -- 172.21.15.111:0/2608493633 shutdown_connections 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9eefe0 0x5610ef9ec870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9e4720 0x5610ef9e06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 --2- 172.21.15.111:0/2608493633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e3e00 0x5610ef9faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 -- 172.21.15.111:0/2608493633 >> 172.21.15.111:0/2608493633 conn(0x5610ef9e0310 msgr2=0x5610ef9e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.767+0000 7f876a395740 1 -- 172.21.15.111:0/2608493633 shutdown_connections 2024-04-24T22:45:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 -- 172.21.15.111:0/2608493633 wait complete. 2024-04-24T22:45:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 -- start start 2024-04-24T22:45:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 Processor -- start 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e3e00 0x5610ef9f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.768+0000 7f876a395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b1f20 con 0x5610ef9e3e00 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f876a395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b3390 con 0x5610ef9eefe0 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f876a395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87601b25f0 con 0x5610ef9e4720 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f876a395740 10 client.0 _ll_register_callbacks cb 0x7f87480d3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f8764a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e3e00 0x5610ef9f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:44108/0 (socket says 172.21.15.111:44108) 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 -- 172.21.15.111:0/2113631731 learned_addr learned my addr 172.21.15.111:0/2113631731 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f8765400640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 -- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 msgr2=0x5610ef9e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 -- 172.21.15.111:0/2113631731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e3e00 msgr2=0x5610ef9f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e3e00 0x5610ef9f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 -- 172.21.15.111:0/2113631731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ef9e97b0 con 0x5610ef9e4720 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f8765400640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f875fe00640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8748004f30 tx=0x7f87480bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b4f87079731ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.769+0000 7f874e000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:45:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874809bad0 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f87480bc0b0 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875800f4e0 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8748005a20 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 10 client.25558 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5610efa65110 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875800f4e0 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8748096ce0 con 0x5610ef9e4720 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 10 client.25558 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 20 client.25558 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610efa65860 0x5610efa67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 4 client.25558 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:45:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.770+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610ef94f340 con 0x5610efa65860 2024-04-24T22:45:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.771+0000 7f8764a00640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610efa65860 0x5610efa67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.771+0000 7f8764a00640 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610efa65860 0x5610efa67c80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00f02f9b2dfde1d server_cookie=8267fef2e9881ae7 in_seq=0 out_seq=0 2024-04-24T22:45:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.771+0000 7f874e000640 10 client.25558 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f874e000640 1 -- 172.21.15.111:0/2113631731 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610ef94f340 con 0x5610efa65860 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f874e000640 10 client.25558 handle_command_reply: tid=0 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 client.25558 shutdown 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 msgr2=0x5610ef9e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8748004f30 tx=0x7f87480bc040 comp rx=0 tx=0).stop 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 shutdown_connections 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5610efa65860 0x5610efa67c80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610ef9eefe0 0x5610ef9e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5610ef9e4720 0x5610ef9e8a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f874e000640 0 client.25558 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 --2- 172.21.15.111:0/2113631731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610ef9e3e00 0x5610ef9f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 >> 172.21.15.111:0/2113631731 conn(0x5610ef9e0310 msgr2=0x5610ef9e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 shutdown_connections 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 1 -- 172.21.15.111:0/2113631731 wait complete. 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.786+0000 7f876a395740 20 client.25558 trim_cache size 0 max 16384 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:45:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:45:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:45:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:45:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 msgr2=0x7f87400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 0x7f87400a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8758009530 tx=0x7f8758009260 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 msgr2=0x7f876019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f874800ebc0 tx=0x7f874800c6a0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 shutdown_connections 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87601a2e70 0x7f87601a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f876010d070 0x7f876019eb40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f87400a2da0 0x7f87400a5260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 --2- 172.21.15.111:0/1102522046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8760107fa0 0x7f876019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 >> 172.21.15.111:0/1102522046 conn(0x7f8760075cd0 msgr2=0x7f8760077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.787+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 shutdown_connections 2024-04-24T22:45:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:17.788+0000 7f8767400640 1 -- 172.21.15.111:0/1102522046 wait complete. 2024-04-24T22:45:18.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:17 smithi186 ceph-mon[32753]: pgmap v103: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:18.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:17 smithi111 ceph-mon[27746]: pgmap v103: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:18.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:17 smithi171 ceph-mon[32337]: pgmap v103: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:20.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:19 smithi186 ceph-mon[32753]: pgmap v104: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:20.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:19 smithi111 ceph-mon[27746]: pgmap v104: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:20.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:19 smithi171 ceph-mon[32337]: pgmap v104: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:22.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:21 smithi186 ceph-mon[32753]: pgmap v105: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:22.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:21 smithi111 ceph-mon[27746]: pgmap v105: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:22.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:21 smithi171 ceph-mon[32337]: pgmap v105: 97 pgs: 97 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:23 smithi186 ceph-mon[32753]: pgmap v106: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-24T22:45:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:23 smithi111 ceph-mon[27746]: pgmap v106: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-24T22:45:24.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:23 smithi171 ceph-mon[32337]: pgmap v106: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-24T22:45:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:25 smithi186 ceph-mon[32753]: pgmap v107: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:25 smithi111 ceph-mon[27746]: pgmap v107: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:25 smithi171 ceph-mon[32337]: pgmap v107: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:64/600: open fd = 3 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:65/600: open fd = 3 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:66/600: open fd = 3 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:67/600: open fd = 3 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:68/600: open fd = 3 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:69/600: open fd = 3 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.573 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:70/600: open fd = 3 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:71/600: open fd = 3 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:72/600: open fd = 3 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.0.smithi111.stdout:73/600: open fd = 3 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:74/600: open fd = 3 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:75/600: open fd = 3 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:76/600: open fd = 3 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.0.smithi111.stdout:77/600: open fd = 3 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:78/600: open fd = 3 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:79/600: open fd = 3 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:80/600: open fd = 3 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.0.smithi111.stdout:81/600: open fd = 3 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:82/600: open fd = 3 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:83/600: open fd = 3 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:84/600: open fd = 3 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.577 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:85/600: open fd = 3 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:86/600: open fd = 3 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:87/600: open fd = 3 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:88/600: open fd = 3 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:89/600: open fd = 3 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:90/600: open fd = 3 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:91/600: open fd = 3 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:92/600: open fd = 3 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:93/600: open fd = 3 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:94/600: open fd = 3 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:95/600: open fd = 3 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:96/600: open fd = 3 2024-04-24T22:45:26.580 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:97/600: open fd = 3 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:98/600: open fd = 3 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:99/600: open fd = 3 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:100/600: open fd = 3 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:101/600: open fd = 3 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:102/600: open fd = 3 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:103/600: open fd = 3 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:104/600: open fd = 3 2024-04-24T22:45:26.582 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:105/600: open fd = 3 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:106/600: open fd = 3 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:107/600: open fd = 3 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.583 INFO:tasks.workunit.client.0.smithi111.stdout:108/600: open fd = 3 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:109/600: open fd = 3 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:110/600: open fd = 3 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:111/600: open fd = 3 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.0.smithi111.stdout:112/600: open fd = 3 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:113/600: open fd = 3 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:114/600: open fd = 3 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:115/600: open fd = 3 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.585 INFO:tasks.workunit.client.0.smithi111.stdout:116/600: open fd = 3 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:117/600: open fd = 3 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:118/600: open fd = 3 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:119/600: open fd = 3 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.586 INFO:tasks.workunit.client.0.smithi111.stdout:120/600: open fd = 3 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:121/600: open fd = 3 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:122/600: open fd = 3 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:123/600: open fd = 3 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:124/600: open fd = 3 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:125/600: open fd = 3 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:126/600: open fd = 3 2024-04-24T22:45:26.588 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:45:28.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:27 smithi186 ceph-mon[32753]: pgmap v108: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:28.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:27 smithi111 ceph-mon[27746]: pgmap v108: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:28.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:27 smithi171 ceph-mon[32337]: pgmap v108: 97 pgs: 97 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-24T22:45:30.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:29 smithi186 ceph-mon[32753]: pgmap v109: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-24T22:45:30.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:29 smithi111 ceph-mon[27746]: pgmap v109: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-24T22:45:30.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:29 smithi171 ceph-mon[32337]: pgmap v109: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-24T22:45:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 -- 172.21.15.111:0/3977189666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60101750 msgr2=0x7fef60101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 --2- 172.21.15.111:0/3977189666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60101750 0x7fef60101bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009930 tx=0x7fef4c02a480 comp rx=0 tx=0).stop 2024-04-24T22:45:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 -- 172.21.15.111:0/3977189666 shutdown_connections 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 --2- 172.21.15.111:0/3977189666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60102110 0x7fef6010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 --2- 172.21.15.111:0/3977189666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60101750 0x7fef60101bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 --2- 172.21.15.111:0/3977189666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60107760 0x7fef60107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 -- 172.21.15.111:0/3977189666 >> 172.21.15.111:0/3977189666 conn(0x7fef60078bd0 msgr2=0x7fef60078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 -- 172.21.15.111:0/3977189666 shutdown_connections 2024-04-24T22:45:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.930+0000 7fef67000640 1 -- 172.21.15.111:0/3977189666 wait complete. 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.931+0000 7fef67000640 1 Processor -- start 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.931+0000 7fef67000640 1 -- start start 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60101750 0x7fef60073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60102110 0x7fef60073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef60111c80 con 0x7fef60107760 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef60111b00 con 0x7fef60102110 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef67000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef60111e00 con 0x7fef60101750 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef65000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef65000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56744/0 (socket says 172.21.15.111:56744) 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef65000640 1 -- 172.21.15.111:0/3772935728 learned_addr learned my addr 172.21.15.111:0/3772935728 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60101750 0x7fef60073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.932+0000 7fef5f400640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60102110 0x7fef60073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 -- 172.21.15.111:0/3772935728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60101750 msgr2=0x7fef60073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60101750 0x7fef60073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 -- 172.21.15.111:0/3772935728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60102110 msgr2=0x7fef60073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60102110 0x7fef60073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 -- 172.21.15.111:0/3772935728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601a6640 con 0x7fef60107760 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef65000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fef5800b9e0 tx=0x7fef5800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef5cc00640 1 -- 172.21.15.111:0/3772935728 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58019070 con 0x7fef60107760 2024-04-24T22:45:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef5cc00640 1 -- 172.21.15.111:0/3772935728 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef58014440 con 0x7fef60107760 2024-04-24T22:45:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef601a6930 con 0x7fef60107760 2024-04-24T22:45:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef5cc00640 1 -- 172.21.15.111:0/3772935728 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580075d0 con 0x7fef60107760 2024-04-24T22:45:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.933+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601a6e50 con 0x7fef60107760 2024-04-24T22:45:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.936+0000 7fef69d7a740 1 Processor -- start 2024-04-24T22:45:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef5cc00640 1 -- 172.21.15.111:0/3772935728 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fef5801c070 con 0x7fef60107760 2024-04-24T22:45:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 -- start start 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 0x55e485c3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c44fe0 0x55e485c473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e485b9dab0 con 0x55e485c3a720 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e485b46640 con 0x55e485c39e00 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef69d7a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e485b99a70 con 0x55e485c44fe0 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.937+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 0x55e485c3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56756/0 (socket says 172.21.15.111:56756) 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c44fe0 0x55e485c473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 -- 172.21.15.111:0/541226907 learned_addr learned my addr 172.21.15.111:0/541226907 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5cc00640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 0x7fef440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 -- 172.21.15.111:0/541226907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c44fe0 msgr2=0x55e485c473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c44fe0 0x55e485c473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 -- 172.21.15.111:0/541226907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 msgr2=0x55e485c3a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 0x55e485c3a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 -- 172.21.15.111:0/541226907 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4858b2210 con 0x55e485c3a720 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5f400640 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 0x55e485c3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5cc00640 1 -- 172.21.15.111:0/3772935728 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fef580ce990 con 0x7fef60107760 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef65000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 0x7fef440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.938+0000 7fef5fe00640 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef50009a40 tx=0x7fef50009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c180666f238bc00f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef55600640 1 -- 172.21.15.111:0/541226907 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50017070 con 0x55e485c3a720 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef55600640 1 -- 172.21.15.111:0/541226907 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50013470 con 0x55e485c3a720 2024-04-24T22:45:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef55600640 1 -- 172.21.15.111:0/541226907 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50012630 con 0x55e485c3a720 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef69d7a740 1 -- 172.21.15.111:0/541226907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 msgr2=0x55e485c44aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef65000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 0x7fef440a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef58009970 tx=0x7fef58014cc0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.939+0000 7fef69d7a740 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef50009a40 tx=0x7fef50009f00 comp rx=0 tx=0).stop 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 -- 172.21.15.111:0/541226907 shutdown_connections 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c44fe0 0x55e485c473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c3a720 0x55e485c44aa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 --2- 172.21.15.111:0/541226907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c39e00 0x55e485c3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 -- 172.21.15.111:0/541226907 >> 172.21.15.111:0/541226907 conn(0x55e485c36310 msgr2=0x55e485c366f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 -- 172.21.15.111:0/541226907 shutdown_connections 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 -- 172.21.15.111:0/541226907 wait complete. 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 -- start start 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.940+0000 7fef69d7a740 1 Processor -- start 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c39e00 0x55e485c41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c44fe0 0x55e485c429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e485c47c10 con 0x55e485c44fe0 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e485b59990 con 0x55e485c3a720 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e485c47d90 con 0x55e485c39e00 2024-04-24T22:45:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef69d7a740 10 client.0 _ll_register_callbacks cb 0x55e485b5b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:53674/0 (socket says 172.21.15.111:53674) 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef5fe00640 1 -- 172.21.15.111:0/3710531043 learned_addr learned my addr 172.21.15.111:0/3710531043 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef5f400640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c39e00 0x55e485c41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.941+0000 7fef5f400640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c44fe0 0x55e485c429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 -- 172.21.15.111:0/3710531043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c39e00 msgr2=0x55e485c41f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c39e00 0x55e485c41f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 -- 172.21.15.111:0/3710531043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c44fe0 msgr2=0x55e485c429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c44fe0 0x55e485c429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 -- 172.21.15.111:0/3710531043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48597e620 con 0x55e485c3a720 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef5000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f865f32178e63d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.942+0000 7fef4a000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50017050 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef50013470 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fef5001c450 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef50013a30 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef69d7a740 10 client.25556 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.945+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e485c53ac0 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50013470 con 0x55e485c3a720 2024-04-24T22:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef50026680 con 0x55e485c3a720 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 10 client.25556 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 20 client.25556 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485c54ed0 0x55e485c572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 4 client.25556 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.946+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e485baaff0 con 0x55e485c54ed0 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.947+0000 7fef5f400640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485c54ed0 0x55e485c572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.948+0000 7fef5f400640 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485c54ed0 0x55e485c572c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e0623bd3217cb8 server_cookie=cdd7867fca23c4a2 in_seq=0 out_seq=0 2024-04-24T22:45:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.948+0000 7fef4a000640 10 client.25556 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.949+0000 7fef4a000640 1 -- 172.21.15.111:0/3710531043 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e485baaff0 con 0x55e485c54ed0 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.949+0000 7fef4a000640 10 client.25556 handle_command_reply: tid=0 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 client.25556 shutdown 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 msgr2=0x55e485c424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef5000ed70 comp rx=0 tx=0).stop 2024-04-24T22:45:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 shutdown_connections 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485c54ed0 0x55e485c572c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c44fe0 0x55e485c429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c3a720 0x55e485c424a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3710531043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c39e00 0x55e485c41f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef4a000640 0 client.25556 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 >> 172.21.15.111:0/3710531043 conn(0x55e485c36310 msgr2=0x55e485c3c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 shutdown_connections 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 1 -- 172.21.15.111:0/3710531043 wait complete. 2024-04-24T22:45:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.950+0000 7fef69d7a740 20 client.25556 trim_cache size 0 max 16384 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 Processor -- start 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 -- start start 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c50ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485c43ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef601b1f30 con 0x55e485c39e00 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef601af800 con 0x55e485c44fe0 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.968+0000 7fef69d7a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef601ad650 con 0x55e485c3a720 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:59198/0 (socket says 172.21.15.111:59198) 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 -- 172.21.15.111:0/2803188845 learned_addr learned my addr 172.21.15.111:0/2803188845 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5f400640 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c50ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5f400640 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485c43ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 -- 172.21.15.111:0/2803188845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 msgr2=0x55e485c43ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485c43ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 -- 172.21.15.111:0/2803188845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 msgr2=0x55e485c50ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c50ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 -- 172.21.15.111:0/2803188845 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e485c31620 con 0x55e485c3a720 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.969+0000 7fef5fe00640 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef50026c40 tx=0x7fef50009270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce02c6fb78a39b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef54c00640 1 -- 172.21.15.111:0/2803188845 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50033020 con 0x55e485c3a720 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef54c00640 1 -- 172.21.15.111:0/2803188845 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50004040 con 0x55e485c3a720 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef54c00640 1 -- 172.21.15.111:0/2803188845 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50017040 con 0x55e485c3a720 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 -- 172.21.15.111:0/2803188845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 msgr2=0x55e485c35c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef50026c40 tx=0x7fef50009270 comp rx=0 tx=0).stop 2024-04-24T22:45:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 -- 172.21.15.111:0/2803188845 shutdown_connections 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485c43ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c35c00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 --2- 172.21.15.111:0/2803188845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c50ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.970+0000 7fef69d7a740 1 -- 172.21.15.111:0/2803188845 >> 172.21.15.111:0/2803188845 conn(0x55e485c36310 msgr2=0x55e485c43790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.971+0000 7fef69d7a740 1 -- 172.21.15.111:0/2803188845 shutdown_connections 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.971+0000 7fef69d7a740 1 -- 172.21.15.111:0/2803188845 wait complete. 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.971+0000 7fef69d7a740 1 -- start start 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.971+0000 7fef69d7a740 1 Processor -- start 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c48e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c49340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485cba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef601adcc0 con 0x55e485c39e00 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef601af130 con 0x55e485c44fe0 2024-04-24T22:45:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef601ae390 con 0x55e485c3a720 2024-04-24T22:45:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef69d7a740 10 client.0 _ll_register_callbacks cb 0x7fef4c0095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef5f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c48e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef5f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c48e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56780/0 (socket says 172.21.15.111:56780) 2024-04-24T22:45:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef5f400640 1 -- 172.21.15.111:0/3959304523 learned_addr learned my addr 172.21.15.111:0/3959304523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.972+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c49340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5f400640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485cba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 -- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 msgr2=0x55e485cba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485cba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 -- 172.21.15.111:0/3959304523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 msgr2=0x55e485c48e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c48e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 -- 172.21.15.111:0/3959304523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e485c36140 con 0x55e485c3a720 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5f400640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef5fe00640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c49340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e485c36bc0 tx=0x7fef50013f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c0892ac5d960d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef4a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50033040 con 0x55e485c3a720 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fef500041f0 con 0x55e485c3a720 2024-04-24T22:45:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.973+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef50004040 con 0x55e485c3a720 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef50017040 con 0x55e485c3a720 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 10 client.25573 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e485cbb600 con 0x55e485c3a720 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50004040 con 0x55e485c3a720 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef500038b0 con 0x55e485c3a720 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 10 client.25573 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 20 client.25573 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485cbbb90 0x55e485cbdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 4 client.25573 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.974+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e485ba5340 con 0x55e485cbbb90 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.975+0000 7fef5f400640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485cbbb90 0x55e485cbdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.975+0000 7fef4a000640 10 client.25573 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.975+0000 7fef5f400640 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485cbbb90 0x55e485cbdfb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dcf346ba5d0218 server_cookie=9b2d8156a45952de in_seq=0 out_seq=0 2024-04-24T22:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef4a000640 1 -- 172.21.15.111:0/3959304523 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e485ba5340 con 0x55e485cbbb90 2024-04-24T22:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef4a000640 10 client.25573 handle_command_reply: tid=0 2024-04-24T22:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 client.25573 shutdown 2024-04-24T22:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 msgr2=0x55e485c49340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c49340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e485c36bc0 tx=0x7fef50013f40 comp rx=0 tx=0).stop 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 shutdown_connections 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55e485cbbb90 0x55e485cbdfb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e485c44fe0 0x55e485cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e485c3a720 0x55e485c49340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 --2- 172.21.15.111:0/3959304523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e485c39e00 0x55e485c48e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef4a000640 0 client.25573 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.990+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 >> 172.21.15.111:0/3959304523 conn(0x55e485c36310 msgr2=0x55e485c42790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.991+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 shutdown_connections 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.991+0000 7fef69d7a740 1 -- 172.21.15.111:0/3959304523 wait complete. 2024-04-24T22:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.991+0000 7fef69d7a740 20 client.25573 trim_cache size 0 max 16384 2024-04-24T22:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 msgr2=0x7fef440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 0x7fef440a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef58009970 tx=0x7fef58014cc0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 msgr2=0x7fef600744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fef5800b9e0 tx=0x7fef5800bea0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 shutdown_connections 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef60107760 0x7fef600744c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef60102110 0x7fef60073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fef440a2da0 0x7fef440a5260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 --2- 172.21.15.111:0/3772935728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef60101750 0x7fef60073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 >> 172.21.15.111:0/3772935728 conn(0x7fef60078bd0 msgr2=0x7fef600feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 shutdown_connections 2024-04-24T22:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:30.992+0000 7fef67000640 1 -- 172.21.15.111:0/3772935728 wait complete. 2024-04-24T22:45:32.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:31 smithi171 ceph-mon[32337]: pgmap v110: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-24T22:45:32.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:31 smithi186 ceph-mon[32753]: pgmap v110: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-24T22:45:32.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:31 smithi111 ceph-mon[27746]: pgmap v110: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-24T22:45:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:33 smithi186 ceph-mon[32753]: pgmap v111: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-24T22:45:34.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:33 smithi111 ceph-mon[27746]: pgmap v111: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-24T22:45:34.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:33 smithi171 ceph-mon[32337]: pgmap v111: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-24T22:45:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:35 smithi186 ceph-mon[32753]: pgmap v112: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:35 smithi111 ceph-mon[27746]: pgmap v112: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:36.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:35 smithi171 ceph-mon[32337]: pgmap v112: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:37 smithi186 ceph-mon[32753]: pgmap v113: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:38.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:37 smithi111 ceph-mon[27746]: pgmap v113: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:38.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:37 smithi171 ceph-mon[32337]: pgmap v113: 97 pgs: 97 active+clean; 4.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:40.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:39 smithi186 ceph-mon[32753]: pgmap v114: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:40.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:39 smithi111 ceph-mon[27746]: pgmap v114: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:40.408 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:39 smithi171 ceph-mon[32337]: pgmap v114: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:42.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:41 smithi186 ceph-mon[32753]: pgmap v115: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T22:45:42.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:41 smithi111 ceph-mon[27746]: pgmap v115: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T22:45:42.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:41 smithi171 ceph-mon[32337]: pgmap v115: 97 pgs: 97 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T22:45:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 -- 172.21.15.111:0/774926367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108600 msgr2=0x7f8ec8114a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 --2- 172.21.15.111:0/774926367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108600 0x7f8ec8114a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009be0 tx=0x7f8ec002a6d0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 -- 172.21.15.111:0/774926367 shutdown_connections 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 --2- 172.21.15.111:0/774926367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108600 0x7f8ec8114a40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 --2- 172.21.15.111:0/774926367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec8107c80 0x7f8ec81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 --2- 172.21.15.111:0/774926367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec810dc10 0x7f8ec810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 -- 172.21.15.111:0/774926367 >> 172.21.15.111:0/774926367 conn(0x7f8ec8077520 msgr2=0x7f8ec8106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 -- 172.21.15.111:0/774926367 shutdown_connections 2024-04-24T22:45:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.138+0000 7f8ecf000640 1 -- 172.21.15.111:0/774926367 wait complete. 2024-04-24T22:45:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 Processor -- start 2024-04-24T22:45:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 -- start start 2024-04-24T22:45:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec8108600 0x7f8ec819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec810dc10 0x7f8ec81a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.139+0000 7f8ecf000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8117f80 con 0x7f8ec810dc10 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecf000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8117e00 con 0x7f8ec8107c80 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecf000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8118100 con 0x7f8ec8108600 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec810dc10 0x7f8ec81a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52644/0 (socket says 172.21.15.111:52644) 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 -- 172.21.15.111:0/3657703507 learned_addr learned my addr 172.21.15.111:0/3657703507 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecd200640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec8108600 0x7f8ec819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 -- 172.21.15.111:0/3657703507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec8108600 msgr2=0x7f8ec819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec8108600 0x7f8ec819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 -- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec810dc10 msgr2=0x7f8ec81a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec810dc10 0x7f8ec81a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.140+0000 7f8ecdc00640 1 -- 172.21.15.111:0/3657703507 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec81a3390 con 0x7f8ec8107c80 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e9f0 tx=0x7f8ebc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ec6000640 1 -- 172.21.15.111:0/3657703507 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00ccc0 con 0x7f8ec8107c80 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ec6000640 1 -- 172.21.15.111:0/3657703507 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc005500 con 0x7f8ec8107c80 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec81a3680 con 0x7f8ec8107c80 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ec6000640 1 -- 172.21.15.111:0/3657703507 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc010690 con 0x7f8ec8107c80 2024-04-24T22:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.141+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81aaf10 con 0x7f8ec8107c80 2024-04-24T22:45:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.145+0000 7f8ed3398740 1 Processor -- start 2024-04-24T22:45:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.145+0000 7f8ed3398740 1 -- start start 2024-04-24T22:45:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ec6000640 1 -- 172.21.15.111:0/3657703507 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8ebc0046c0 con 0x7f8ec8107c80 2024-04-24T22:45:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7ee190 0x56068c7f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56068c746ab0 con 0x56068c7e2fb0 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56068c6ef640 con 0x56068c7ee190 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ed3398740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56068c742a70 con 0x56068c7e38d0 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39588/0 (socket says 172.21.15.111:39588) 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 -- 172.21.15.111:0/658512341 learned_addr learned my addr 172.21.15.111:0/658512341 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ecd200640 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7ee190 0x56068c7f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 -- 172.21.15.111:0/658512341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 msgr2=0x56068c7edc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7edc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.146+0000 7f8ece600640 1 -- 172.21.15.111:0/658512341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7ee190 msgr2=0x56068c7f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ece600640 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7ee190 0x56068c7f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ece600640 1 -- 172.21.15.111:0/658512341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068c45b210 con 0x56068c7e2fb0 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ece600640 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009be0 tx=0x7f8ec0030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a82402f3080f434 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ec6000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 0x7f8eb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8eb6a00640 1 -- 172.21.15.111:0/658512341 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0041070 con 0x56068c7e2fb0 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8eb6a00640 1 -- 172.21.15.111:0/658512341 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec00307a0 con 0x56068c7e2fb0 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8eb6a00640 1 -- 172.21.15.111:0/658512341 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036040 con 0x56068c7e2fb0 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ec6000640 1 -- 172.21.15.111:0/3657703507 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f8ebc0cf600 con 0x7f8ec8107c80 2024-04-24T22:45:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.147+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 0x7f8eb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 -- 172.21.15.111:0/658512341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 msgr2=0x56068c7e3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009be0 tx=0x7f8ec0030430 comp rx=0 tx=0).stop 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ecdc00640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 0x7f8eb00a51b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0cce30 tx=0x7f8ebc009f50 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 -- 172.21.15.111:0/658512341 shutdown_connections 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7ee190 0x56068c7f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 --2- 172.21.15.111:0/658512341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7e2fb0 0x56068c7e3390 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 -- 172.21.15.111:0/658512341 >> 172.21.15.111:0/658512341 conn(0x56068c7df4c0 msgr2=0x56068c7df8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 -- 172.21.15.111:0/658512341 shutdown_connections 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.148+0000 7f8ed3398740 1 -- 172.21.15.111:0/658512341 wait complete. 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 -- start start 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 Processor -- start 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c7ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56068c7f0d60 con 0x56068c7ee190 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56068c702990 con 0x56068c7e2fb0 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.149+0000 7f8ed3398740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56068c7f0ee0 con 0x56068c7e38d0 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ed3398740 10 client.0 _ll_register_callbacks cb 0x56068c704950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52652/0 (socket says 172.21.15.111:52652) 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 -- 172.21.15.111:0/122575485 learned_addr learned my addr 172.21.15.111:0/122575485 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ecd200640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c7ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 -- 172.21.15.111:0/122575485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 msgr2=0x56068c7eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 -- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 msgr2=0x56068c7eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.150+0000 7f8ece600640 1 -- 172.21.15.111:0/122575485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068c527620 con 0x56068c7ee190 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8ece600640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c7ebb40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004af0 tx=0x7f8ec002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee63c42df6467c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8eab400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036030 con 0x56068c7ee190 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f8ec0003970 con 0x56068c7ee190 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec00307a0 con 0x56068c7ee190 2024-04-24T22:45:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8ec0041040 con 0x56068c7ee190 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8ed3398740 10 client.15501 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56068c7fcbd0 con 0x56068c7ee190 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.151+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec00307a0 con 0x56068c7ee190 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8ec0004370 con 0x56068c7ee190 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 20 client.15501 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c7fdf20 0x56068c800310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 4 client.15501 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56068c753ff0 con 0x56068c7fdf20 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.152+0000 7f8ecd200640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c7fdf20 0x56068c800310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.153+0000 7f8ecd200640 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c7fdf20 0x56068c800310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e31fb99ee3441cb server_cookie=396dc154c3a91fc7 in_seq=0 out_seq=0 2024-04-24T22:45:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.153+0000 7f8eab400640 10 client.15501 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8eab400640 1 -- 172.21.15.111:0/122575485 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56068c753ff0 con 0x56068c7fdf20 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8eab400640 10 client.15501 handle_command_reply: tid=0 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 client.15501 shutdown 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 msgr2=0x56068c7ebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c7ebb40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004af0 tx=0x7f8ec002a6d0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 shutdown_connections 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c7fdf20 0x56068c800310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c7ebb40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c7eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8eab400640 0 client.15501 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 --2- 172.21.15.111:0/122575485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 >> 172.21.15.111:0/122575485 conn(0x56068c7df4c0 msgr2=0x56068c7e51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 shutdown_connections 2024-04-24T22:45:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 1 -- 172.21.15.111:0/122575485 wait complete. 2024-04-24T22:45:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.154+0000 7f8ed3398740 20 client.15501 trim_cache size 0 max 16384 2024-04-24T22:45:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.170+0000 7f8ed3398740 1 Processor -- start 2024-04-24T22:45:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.170+0000 7f8ed3398740 1 -- start start 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c803330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b6060 con 0x56068c7ee190 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b3930 con 0x56068c7e2fb0 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ed3398740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b1780 con 0x56068c7e38d0 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39598/0 (socket says 172.21.15.111:39598) 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 -- 172.21.15.111:0/2273349495 learned_addr learned my addr 172.21.15.111:0/2273349495 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ecd200640 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c803330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 -- 172.21.15.111:0/2273349495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 msgr2=0x56068c803330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c803330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 -- 172.21.15.111:0/2273349495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 msgr2=0x56068c7f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.171+0000 7f8ece600640 1 -- 172.21.15.111:0/2273349495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068c7f0db0 con 0x56068c7ee190 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.172+0000 7f8ece600640 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0003f30 tx=0x7f8ec0033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505092eaf27a11e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.172+0000 7f8eb6000640 1 -- 172.21.15.111:0/2273349495 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036020 con 0x56068c7ee190 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.172+0000 7f8eb6000640 1 -- 172.21.15.111:0/2273349495 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0003970 con 0x56068c7ee190 2024-04-24T22:45:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.172+0000 7f8eb6000640 1 -- 172.21.15.111:0/2273349495 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0030eb0 con 0x56068c7ee190 2024-04-24T22:45:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- 172.21.15.111:0/2273349495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 msgr2=0x56068c803870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0003f30 tx=0x7f8ec0033040 comp rx=0 tx=0).stop 2024-04-24T22:45:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- 172.21.15.111:0/2273349495 shutdown_connections 2024-04-24T22:45:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c803870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c803330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2273349495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c7f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- 172.21.15.111:0/2273349495 >> 172.21.15.111:0/2273349495 conn(0x56068c7df4c0 msgr2=0x56068c7f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- 172.21.15.111:0/2273349495 shutdown_connections 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- 172.21.15.111:0/2273349495 wait complete. 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.173+0000 7f8ed3398740 1 -- start start 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 Processor -- start 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c8014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b1df0 con 0x56068c7ee190 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b3260 con 0x56068c7e2fb0 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b24c0 con 0x56068c7e38d0 2024-04-24T22:45:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ed3398740 10 client.0 _ll_register_callbacks cb 0x7f8ec0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.174+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ece600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39608/0 (socket says 172.21.15.111:39608) 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ece600640 1 -- 172.21.15.111:0/2930402740 learned_addr learned my addr 172.21.15.111:0/2930402740 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ece600640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c8014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 -- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 msgr2=0x56068c8014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c8014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 -- 172.21.15.111:0/2930402740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 msgr2=0x56068c863100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 -- 172.21.15.111:0/2930402740 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068c8636f0 con 0x56068c7e38d0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ece600640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8ecd200640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c801a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56068c7ea920 tx=0x7f8eb801f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a42f4527a8bd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.175+0000 7f8eab400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8004440 con 0x56068c7e38d0 2024-04-24T22:45:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec0003970 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f8eb8010960 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8eb8010b60 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 10 client.25591 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56068c864a90 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0003970 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8eb8027430 con 0x56068c7e38d0 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 10 client.25591 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 20 client.25591 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c8651c0 0x56068c8675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 4 client.25591 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:45:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.176+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56068c74e340 con 0x56068c8651c0 2024-04-24T22:45:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.177+0000 7f8ece600640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c8651c0 0x56068c8675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.177+0000 7f8ece600640 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c8651c0 0x56068c8675e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b95b05c1284ae30 server_cookie=56ff44b845f5a6ab in_seq=0 out_seq=0 2024-04-24T22:45:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.177+0000 7f8eab400640 10 client.25591 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.192+0000 7f8eab400640 1 -- 172.21.15.111:0/2930402740 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56068c74e340 con 0x56068c8651c0 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.192+0000 7f8eab400640 10 client.25591 handle_command_reply: tid=0 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.192+0000 7f8ed3398740 1 client.25591 shutdown 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.192+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 msgr2=0x56068c801a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.192+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c801a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56068c7ea920 tx=0x7f8eb801f770 comp rx=0 tx=0).stop 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 shutdown_connections 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56068c8651c0 0x56068c8675e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56068c7ee190 0x56068c863100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56068c7e38d0 0x56068c801a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8eab400640 0 client.25591 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 --2- 172.21.15.111:0/2930402740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56068c7e2fb0 0x56068c8014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 >> 172.21.15.111:0/2930402740 conn(0x56068c7df4c0 msgr2=0x56068c7eb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 shutdown_connections 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 1 -- 172.21.15.111:0/2930402740 wait complete. 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.193+0000 7f8ed3398740 20 client.25591 trim_cache size 0 max 16384 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:45:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 msgr2=0x7f8eb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 0x7f8eb00a51b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0cce30 tx=0x7f8ebc009f50 comp rx=0 tx=0).stop 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 msgr2=0x7f8ec819e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e9f0 tx=0x7f8ebc00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 shutdown_connections 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec810dc10 0x7f8ec81a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8eb00a2cf0 0x7f8eb00a51b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec8108600 0x7f8ec819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 --2- 172.21.15.111:0/3657703507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8107c80 0x7f8ec819e3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 >> 172.21.15.111:0/3657703507 conn(0x7f8ec8077520 msgr2=0x7f8ec80784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 shutdown_connections 2024-04-24T22:45:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:44.194+0000 7f8ecf000640 1 -- 172.21.15.111:0/3657703507 wait complete. 2024-04-24T22:45:44.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:43 smithi186 ceph-mon[32753]: pgmap v116: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 17 op/s 2024-04-24T22:45:44.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:43 smithi111 ceph-mon[27746]: pgmap v116: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 17 op/s 2024-04-24T22:45:44.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:43 smithi171 ceph-mon[32337]: pgmap v116: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 17 op/s 2024-04-24T22:45:46.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:45 smithi186 ceph-mon[32753]: pgmap v117: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:46.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:45 smithi111 ceph-mon[27746]: pgmap v117: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:46.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:45 smithi171 ceph-mon[32337]: pgmap v117: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:48.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:47 smithi186 ceph-mon[32753]: pgmap v118: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:48.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:47 smithi111 ceph-mon[27746]: pgmap v118: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:48.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:47 smithi171 ceph-mon[32337]: pgmap v118: 97 pgs: 97 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T22:45:50.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:49 smithi186 ceph-mon[32753]: pgmap v119: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:50.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:49 smithi111 ceph-mon[27746]: pgmap v119: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:50.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:49 smithi171 ceph-mon[32337]: pgmap v119: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-24T22:45:52.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:51 smithi186 ceph-mon[32753]: pgmap v120: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:52.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:51 smithi111 ceph-mon[27746]: pgmap v120: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:52.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:51 smithi171 ceph-mon[32337]: pgmap v120: 97 pgs: 97 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:45:54.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:54 smithi111 ceph-mon[27746]: pgmap v121: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-24T22:45:54.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:54 smithi171 ceph-mon[32337]: pgmap v121: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-24T22:45:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:54 smithi186 ceph-mon[32753]: pgmap v121: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-24T22:45:56.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:56 smithi111 ceph-mon[27746]: pgmap v122: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:45:56.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:56 smithi171 ceph-mon[32337]: pgmap v122: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:45:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:56 smithi186 ceph-mon[32753]: pgmap v122: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 -- 172.21.15.111:0/1420070268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd130107c80 msgr2=0x7fd1301080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 --2- 172.21.15.111:0/1420070268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd130107c80 0x7fd1301080c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd118009930 tx=0x7fd11802a440 comp rx=0 tx=0).stop 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 -- 172.21.15.111:0/1420070268 shutdown_connections 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 --2- 172.21.15.111:0/1420070268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130108600 0x7fd130114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 --2- 172.21.15.111:0/1420070268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd130107c80 0x7fd1301080c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 --2- 172.21.15.111:0/1420070268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd13010dc10 0x7fd13010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 -- 172.21.15.111:0/1420070268 >> 172.21.15.111:0/1420070268 conn(0x7fd130077520 msgr2=0x7fd130106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 -- 172.21.15.111:0/1420070268 shutdown_connections 2024-04-24T22:45:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.335+0000 7fd135200640 1 -- 172.21.15.111:0/1420070268 wait complete. 2024-04-24T22:45:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 Processor -- start 2024-04-24T22:45:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 -- start start 2024-04-24T22:45:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd130108600 0x7fd13019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 0x7fd1301a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd130117fd0 con 0x7fd13010dc10 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd130117e50 con 0x7fd130108600 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.336+0000 7fd135200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd130118150 con 0x7fd130107c80 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 0x7fd1301a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:38626/0 (socket says 172.21.15.111:38626) 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 -- 172.21.15.111:0/2319429743 learned_addr learned my addr 172.21.15.111:0/2319429743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd126a00640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd130108600 0x7fd13019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 -- 172.21.15.111:0/2319429743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd130108600 msgr2=0x7fd13019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd130108600 0x7fd13019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 -- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 msgr2=0x7fd1301a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 0x7fd1301a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 -- 172.21.15.111:0/2319429743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1301a3430 con 0x7fd130107c80 2024-04-24T22:45:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12fe00640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 0x7fd1301a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:45:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.337+0000 7fd12f400640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e9f0 tx=0x7fd12000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.338+0000 7fd12cc00640 1 -- 172.21.15.111:0/2319429743 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12000ccc0 con 0x7fd130107c80 2024-04-24T22:45:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.338+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1301a3720 con 0x7fd130107c80 2024-04-24T22:45:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.338+0000 7fd12cc00640 1 -- 172.21.15.111:0/2319429743 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120010910 con 0x7fd130107c80 2024-04-24T22:45:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.338+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1301aafb0 con 0x7fd130107c80 2024-04-24T22:45:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.338+0000 7fd12cc00640 1 -- 172.21.15.111:0/2319429743 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120018880 con 0x7fd130107c80 2024-04-24T22:45:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.341+0000 7fd12cc00640 1 -- 172.21.15.111:0/2319429743 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd120010430 con 0x7fd130107c80 2024-04-24T22:45:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.341+0000 7fd139764740 1 Processor -- start 2024-04-24T22:45:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.341+0000 7fd139764740 1 -- start start 2024-04-24T22:45:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bd7190 0x55c325bd9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c325b2fab0 con 0x55c325bcbfb0 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c325ad8640 con 0x55c325bd7190 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd139764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c325b2ba70 con 0x55c325bcc8d0 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47796/0 (socket says 172.21.15.111:47796) 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 -- 172.21.15.111:0/1620699260 learned_addr learned my addr 172.21.15.111:0/1620699260 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12f400640 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd126a00640 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bd7190 0x55c325bd9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 -- 172.21.15.111:0/1620699260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 msgr2=0x55c325bd6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 -- 172.21.15.111:0/1620699260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bd7190 msgr2=0x55c325bd9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bd7190 0x55c325bd9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.342+0000 7fd12fe00640 1 -- 172.21.15.111:0/1620699260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c325844210 con 0x55c325bcbfb0 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd12fe00640 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd128011810 tx=0x7fd128011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d067508d1c18d604 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd12cc00640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 0x7fd1140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd124c00640 1 -- 172.21.15.111:0/1620699260 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128011310 con 0x55c325bcbfb0 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd124c00640 1 -- 172.21.15.111:0/1620699260 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12800f9a0 con 0x55c325bcbfb0 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd124c00640 1 -- 172.21.15.111:0/1620699260 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128005730 con 0x55c325bcbfb0 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd12cc00640 1 -- 172.21.15.111:0/2319429743 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fd1200cf830 con 0x7fd130107c80 2024-04-24T22:45:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.343+0000 7fd12f400640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 0x7fd1140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd12f400640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 0x7fd1140a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c325bcd550 tx=0x7fd1200d5530 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 -- 172.21.15.111:0/1620699260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 msgr2=0x55c325bcc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd128011810 tx=0x7fd128011cd0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 -- 172.21.15.111:0/1620699260 shutdown_connections 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bd7190 0x55c325bd9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 --2- 172.21.15.111:0/1620699260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcbfb0 0x55c325bcc390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 -- 172.21.15.111:0/1620699260 >> 172.21.15.111:0/1620699260 conn(0x55c325bc84c0 msgr2=0x55c325bc88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 -- 172.21.15.111:0/1620699260 shutdown_connections 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.344+0000 7fd139764740 1 -- 172.21.15.111:0/1620699260 wait complete. 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.345+0000 7fd139764740 1 -- start start 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.345+0000 7fd139764740 1 Processor -- start 2024-04-24T22:45:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.345+0000 7fd139764740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bd4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.345+0000 7fd139764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd139764740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 0x55c325bd4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd139764740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c325bd9dc0 con 0x55c325bd7190 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd139764740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c325aeb990 con 0x55c325bcbfb0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd139764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c325bd9f40 con 0x55c325bcc8d0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd139764740 10 client.0 _ll_register_callbacks cb 0x55c325aed950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 0x55c325bd4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 0x55c325bd4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47798/0 (socket says 172.21.15.111:47798) 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd12fe00640 1 -- 172.21.15.111:0/2672145411 learned_addr learned my addr 172.21.15.111:0/2672145411 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 -- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 msgr2=0x55c325bd4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd12fe00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bd4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bd4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 -- 172.21.15.111:0/2672145411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 msgr2=0x55c325bd4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 0x55c325bd4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd126a00640 1 -- 172.21.15.111:0/2672145411 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c325910620 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.346+0000 7fd12fe00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd126a00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd4650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd11802a960 tx=0x7fd1180043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42815d836c73222 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd11d600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118035040 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fd118005450 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd12800f9a0 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd118005650 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd139764740 10 client.25603 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.347+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c325be5c70 con 0x55c325bcc8d0 2024-04-24T22:45:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12800f9a0 con 0x55c325bcc8d0 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd118003990 con 0x55c325bcc8d0 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 10 client.25603 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 20 client.25603 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325be7130 0x55c325be9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 4 client.25603 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c325b3cff0 con 0x55c325be7130 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.348+0000 7fd12fe00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325be7130 0x55c325be9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.349+0000 7fd12fe00640 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325be7130 0x55c325be9520 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546108bb0b859434 server_cookie=939c5a0d2d4911b6 in_seq=0 out_seq=0 2024-04-24T22:45:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.349+0000 7fd11d600640 10 client.25603 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd11d600640 1 -- 172.21.15.111:0/2672145411 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c325b3cff0 con 0x55c325be7130 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd11d600640 10 client.25603 handle_command_reply: tid=0 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 client.25603 shutdown 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 msgr2=0x55c325bd4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd4650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd11802a960 tx=0x7fd1180043b0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 shutdown_connections 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325be7130 0x55c325be9520 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bd7190 0x55c325bd4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bcc8d0 0x55c325bd4650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 --2- 172.21.15.111:0/2672145411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd11d600640 0 client.25603 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 >> 172.21.15.111:0/2672145411 conn(0x55c325bc84c0 msgr2=0x55c325bce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 shutdown_connections 2024-04-24T22:45:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.350+0000 7fd139764740 1 -- 172.21.15.111:0/2672145411 wait complete. 2024-04-24T22:45:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.351+0000 7fd139764740 20 client.25603 trim_cache size 0 max 16384 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.366+0000 7fd139764740 1 Processor -- start 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 -- start start 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325be2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325beedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b6140 con 0x55c325bcc8d0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b3a10 con 0x55c325bcbfb0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd139764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b1860 con 0x55c325bd7190 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47804/0 (socket says 172.21.15.111:47804) 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 -- 172.21.15.111:0/1109263410 learned_addr learned my addr 172.21.15.111:0/1109263410 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd12fe00640 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325be2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd12fe00640 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325beedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 -- 172.21.15.111:0/1109263410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 msgr2=0x55c325beedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325beedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 -- 172.21.15.111:0/1109263410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 msgr2=0x55c325be2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325be2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.367+0000 7fd126a00640 1 -- 172.21.15.111:0/1109263410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c325bd97b0 con 0x55c325bcc8d0 2024-04-24T22:45:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.368+0000 7fd126a00640 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd118031550 tx=0x7fd118005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c108c94d4baa993 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.368+0000 7fd11fe00640 1 -- 172.21.15.111:0/1109263410 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118035030 con 0x55c325bcc8d0 2024-04-24T22:45:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.368+0000 7fd11fe00640 1 -- 172.21.15.111:0/1109263410 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd118041070 con 0x55c325bcc8d0 2024-04-24T22:45:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.368+0000 7fd11fe00640 1 -- 172.21.15.111:0/1109263410 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118053070 con 0x55c325bcc8d0 2024-04-24T22:45:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- 172.21.15.111:0/1109263410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 msgr2=0x55c325bc7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd118031550 tx=0x7fd118005a40 comp rx=0 tx=0).stop 2024-04-24T22:45:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- 172.21.15.111:0/1109263410 shutdown_connections 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325beedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bc7da0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 --2- 172.21.15.111:0/1109263410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325be2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- 172.21.15.111:0/1109263410 >> 172.21.15.111:0/1109263410 conn(0x55c325bc84c0 msgr2=0x55c325bd61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- 172.21.15.111:0/1109263410 shutdown_connections 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- 172.21.15.111:0/1109263410 wait complete. 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 -- start start 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.369+0000 7fd139764740 1 Processor -- start 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bdf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325bdc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b1ed0 con 0x55c325bcc8d0 2024-04-24T22:45:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b3340 con 0x55c325bcbfb0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b25a0 con 0x55c325bd7190 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd139764740 10 client.0 _ll_register_callbacks cb 0x7fd118009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47816/0 (socket says 172.21.15.111:47816) 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 -- 172.21.15.111:0/3246487060 learned_addr learned my addr 172.21.15.111:0/3246487060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd12fe00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325bdc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd12fe00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bdf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 -- 172.21.15.111:0/3246487060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 msgr2=0x55c325bdc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325bdc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 -- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 msgr2=0x55c325bdf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bdf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.370+0000 7fd126a00640 1 -- 172.21.15.111:0/3246487060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c325bdcf10 con 0x55c325bcc8d0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd12fe00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325bdc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd126a00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1180043f0 tx=0x7fd118004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1b6f8b36062173 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd11d600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118053070 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fd118031d60 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd118041070 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd118067050 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd139764740 10 client.15525 fetch_fsmap learned FSMap version 24 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.371+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c325c4e6f0 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd118041070 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd1180048c0 con 0x55c325bcc8d0 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 20 client.15525 populate_metadata read hostname 'smithi111' 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325c4ed50 0x55c325c51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 4 client.15525 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:45:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.372+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c325b37340 con 0x55c325c4ed50 2024-04-24T22:45:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.373+0000 7fd12fe00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325c4ed50 0x55c325c51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.373+0000 7fd12fe00640 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325c4ed50 0x55c325c51170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278456c424c650 server_cookie=2d600265befb53ae in_seq=0 out_seq=0 2024-04-24T22:45:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.373+0000 7fd11d600640 10 client.15525 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd11d600640 1 -- 172.21.15.111:0/3246487060 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c325b37340 con 0x55c325c4ed50 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd11d600640 10 client.15525 handle_command_reply: tid=0 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 client.15525 shutdown 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 msgr2=0x55c325bdc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1180043f0 tx=0x7fd118004580 comp rx=0 tx=0).stop 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 shutdown_connections 2024-04-24T22:45:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55c325c4ed50 0x55c325c51170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c325bd7190 0x55c325bdc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd11d600640 0 client.15525 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c325bcc8d0 0x55c325bdc160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 --2- 172.21.15.111:0/3246487060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c325bcbfb0 0x55c325bdf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.388+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 >> 172.21.15.111:0/3246487060 conn(0x55c325bc84c0 msgr2=0x55c325bd5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.389+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 shutdown_connections 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.389+0000 7fd139764740 1 -- 172.21.15.111:0/3246487060 wait complete. 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.389+0000 7fd139764740 20 client.15525 trim_cache size 0 max 16384 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:45:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:45:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:45:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:45:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 msgr2=0x7fd1140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 0x7fd1140a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c325bcd550 tx=0x7fd1200d5530 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 msgr2=0x7fd13019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e9f0 tx=0x7fd12000edf0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 shutdown_connections 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd13010dc10 0x7fd1301a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd130108600 0x7fd13019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd1140a2cf0 0x7fd1140a51b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 --2- 172.21.15.111:0/2319429743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd130107c80 0x7fd13019e490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 >> 172.21.15.111:0/2319429743 conn(0x7fd130077520 msgr2=0x7fd130078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 shutdown_connections 2024-04-24T22:45:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:45:57.390+0000 7fd135200640 1 -- 172.21.15.111:0/2319429743 wait complete. 2024-04-24T22:45:58.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:45:58 smithi111 ceph-mon[27746]: pgmap v123: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:45:58.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:45:58 smithi171 ceph-mon[32337]: pgmap v123: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:45:58.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:45:58 smithi186 ceph-mon[32753]: pgmap v123: 97 pgs: 97 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-24T22:46:00.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:00 smithi111 ceph-mon[27746]: pgmap v124: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-04-24T22:46:00.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:00 smithi171 ceph-mon[32337]: pgmap v124: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-04-24T22:46:00.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:00 smithi186 ceph-mon[32753]: pgmap v124: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-04-24T22:46:02.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:02 smithi171 ceph-mon[32337]: pgmap v125: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:46:02.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:02 smithi186 ceph-mon[32753]: pgmap v125: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:46:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:02 smithi111 ceph-mon[27746]: pgmap v125: 97 pgs: 97 active+clean; 5.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-24T22:46:04.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:04 smithi171 ceph-mon[32337]: pgmap v126: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-04-24T22:46:04.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:04 smithi186 ceph-mon[32753]: pgmap v126: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-04-24T22:46:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:04 smithi111 ceph-mon[27746]: pgmap v126: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-04-24T22:46:06.302 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:06 smithi186 ceph-mon[32753]: pgmap v127: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:06.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:06 smithi171 ceph-mon[32337]: pgmap v127: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:06 smithi111 ceph-mon[27746]: pgmap v127: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:08.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:08 smithi171 ceph-mon[32337]: pgmap v128: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:08.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:08 smithi186 ceph-mon[32753]: pgmap v128: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:08 smithi111 ceph-mon[27746]: pgmap v128: 97 pgs: 97 active+clean; 5.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-24T22:46:10.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:10 smithi171 ceph-mon[32337]: pgmap v129: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 32 op/s 2024-04-24T22:46:10.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:10 smithi186 ceph-mon[32753]: pgmap v129: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 32 op/s 2024-04-24T22:46:10.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:10 smithi111 ceph-mon[27746]: pgmap v129: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 32 op/s 2024-04-24T22:46:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 -- 172.21.15.111:0/2689514709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888074360 msgr2=0x7f888810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 --2- 172.21.15.111:0/2689514709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888074360 0x7f888810daf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f887002a480 comp rx=0 tx=0).stop 2024-04-24T22:46:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 -- 172.21.15.111:0/2689514709 shutdown_connections 2024-04-24T22:46:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 --2- 172.21.15.111:0/2689514709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f888810e030 0x7f8888110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 --2- 172.21.15.111:0/2689514709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8888074360 0x7f888810daf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 --2- 172.21.15.111:0/2689514709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888073a40 0x7f8888073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 -- 172.21.15.111:0/2689514709 >> 172.21.15.111:0/2689514709 conn(0x7f8888078bd0 msgr2=0x7f8888078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 -- 172.21.15.111:0/2689514709 shutdown_connections 2024-04-24T22:46:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.534+0000 7f888de00640 1 -- 172.21.15.111:0/2689514709 wait complete. 2024-04-24T22:46:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.535+0000 7f888de00640 1 Processor -- start 2024-04-24T22:46:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 -- start start 2024-04-24T22:46:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888073a40 0x7f888819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 0x7f888819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8888113d60 con 0x7f8888074360 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8888113be0 con 0x7f888810e030 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f888de00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8888113ee0 con 0x7f8888073a40 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f887e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f887e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47772/0 (socket says 172.21.15.111:47772) 2024-04-24T22:46:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f887e000640 1 -- 172.21.15.111:0/4145862112 learned_addr learned my addr 172.21.15.111:0/4145862112 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f8886a00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888073a40 0x7f888819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.536+0000 7f8887400640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 0x7f888819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 -- 172.21.15.111:0/4145862112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888073a40 msgr2=0x7f888819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888073a40 0x7f888819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 -- 172.21.15.111:0/4145862112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 msgr2=0x7f888819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 0x7f888819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 -- 172.21.15.111:0/4145862112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819f0f0 con 0x7f8888074360 2024-04-24T22:46:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f8887400640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 0x7f888819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f887e000640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f88700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f888ca00640 1 -- 172.21.15.111:0/4145862112 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887003f030 con 0x7f8888074360 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819f380 con 0x7f8888074360 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f888ca00640 1 -- 172.21.15.111:0/4145862112 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8870003ce0 con 0x7f8888074360 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.537+0000 7f888ca00640 1 -- 172.21.15.111:0/4145862112 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870035070 con 0x7f8888074360 2024-04-24T22:46:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.538+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a6c20 con 0x7f8888074360 2024-04-24T22:46:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 Processor -- start 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 -- start start 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f888ca00640 1 -- 172.21.15.111:0/4145862112 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8870042030 con 0x7f8888074360 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85acb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf85a21ab0 con 0x55bf85abde00 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bf859ca640 con 0x55bf85abe720 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.541+0000 7f8890dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf85a1da70 con 0x55bf85ac8fe0 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8887400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:45186/0 (socket says 172.21.15.111:45186) 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8887400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47788/0 (socket says 172.21.15.111:47788) 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 -- 172.21.15.111:0/1241687388 learned_addr learned my addr 172.21.15.111:0/1241687388 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f887e000640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85acb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 -- 172.21.15.111:0/1241687388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 msgr2=0x55bf85acb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85acb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 -- 172.21.15.111:0/1241687388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 msgr2=0x55bf85abe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 -- 172.21.15.111:0/1241687388 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf85736210 con 0x55bf85abe720 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8887400640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f887e000640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85acb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.542+0000 7f8886a00640 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8878009cf0 tx=0x7f8878011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43edc6954f5af258 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f8877e00640 1 -- 172.21.15.111:0/1241687388 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878004230 con 0x55bf85abe720 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f8877e00640 1 -- 172.21.15.111:0/1241687388 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88780043d0 con 0x55bf85abe720 2024-04-24T22:46:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f8877e00640 1 -- 172.21.15.111:0/1241687388 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878019900 con 0x55bf85abe720 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f888ca00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 0x7f886c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f888ca00640 1 -- 172.21.15.111:0/4145862112 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f88700efa60 con 0x7f8888074360 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.543+0000 7f8887400640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 0x7f886c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- 172.21.15.111:0/1241687388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 msgr2=0x55bf85ac8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8878009cf0 tx=0x7f8878011810 comp rx=0 tx=0).stop 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- 172.21.15.111:0/1241687388 shutdown_connections 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85acb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac8aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1241687388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85abe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- 172.21.15.111:0/1241687388 >> 172.21.15.111:0/1241687388 conn(0x55bf85aba310 msgr2=0x55bf85aba6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8887400640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 0x7f886c0a5260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8880005fd0 tx=0x7f8880009810 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- 172.21.15.111:0/1241687388 shutdown_connections 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- 172.21.15.111:0/1241687388 wait complete. 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 -- start start 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.544+0000 7f8890dcc740 1 Processor -- start 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abde00 0x55bf85ac5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf85acbc10 con 0x55bf85ac8fe0 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bf859dd990 con 0x55bf85abe720 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf85acbd90 con 0x55bf85abde00 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8890dcc740 10 client.0 _ll_register_callbacks cb 0x55bf859df950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8887400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f887e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8887400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47792/0 (socket says 172.21.15.111:47792) 2024-04-24T22:46:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8887400640 1 -- 172.21.15.111:0/2784802524 learned_addr learned my addr 172.21.15.111:0/2784802524 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.545+0000 7f8886a00640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abde00 0x55bf85ac5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 -- 172.21.15.111:0/2784802524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abde00 msgr2=0x55bf85ac5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abde00 0x55bf85ac5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 -- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 msgr2=0x55bf85ac6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 -- 172.21.15.111:0/2784802524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf85802620 con 0x55bf85ac8fe0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8887400640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8880010a50 tx=0x7f8880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b674749255537231 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8874c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888001d8f0 con 0x55bf85ac8fe0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f888001da90 con 0x55bf85ac8fe0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.546+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88780043d0 con 0x55bf85ac8fe0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f888001dc90 con 0x55bf85ac8fe0 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 10 client.15537 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf85ad7aa0 con 0x55bf85ac8fe0 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88780043d0 con 0x55bf85ac8fe0 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8880032490 con 0x55bf85ac8fe0 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 10 client.15537 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 20 client.15537 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85ad8f10 0x55bf85adb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 4 client.15537 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.547+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf85a2eff0 con 0x55bf85ad8f10 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.548+0000 7f8886a00640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85ad8f10 0x55bf85adb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.548+0000 7f8886a00640 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85ad8f10 0x55bf85adb320 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f16a6edf3fa691d server_cookie=c18d8532de14af1f in_seq=0 out_seq=0 2024-04-24T22:46:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.548+0000 7f8874c00640 10 client.15537 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.549+0000 7f8874c00640 1 -- 172.21.15.111:0/2784802524 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf85a2eff0 con 0x55bf85ad8f10 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.549+0000 7f8874c00640 10 client.15537 handle_command_reply: tid=0 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.549+0000 7f8890dcc740 1 client.15537 shutdown 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 msgr2=0x55bf85ac6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8880010a50 tx=0x7f8880004a40 comp rx=0 tx=0).stop 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 shutdown_connections 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85ad8f10 0x55bf85adb320 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85ac8fe0 0x55bf85ac6950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abe720 0x55bf85ac6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8874c00640 0 client.15537 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 --2- 172.21.15.111:0/2784802524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abde00 0x55bf85ac5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 >> 172.21.15.111:0/2784802524 conn(0x55bf85aba310 msgr2=0x55bf85ac0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 shutdown_connections 2024-04-24T22:46:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 1 -- 172.21.15.111:0/2784802524 wait complete. 2024-04-24T22:46:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.550+0000 7f8890dcc740 20 client.15537 trim_cache size 0 max 16384 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 Processor -- start 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 -- start start 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 0x55bf85ac65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85ac8fe0 0x55bf85ae0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b1e00 con 0x55bf85abde00 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88881af6d0 con 0x55bf85ac8fe0 2024-04-24T22:46:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.566+0000 7f8890dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ad520 con 0x55bf85abe720 2024-04-24T22:46:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f8886a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f8887400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85ac8fe0 0x55bf85ae0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f8886a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47806/0 (socket says 172.21.15.111:47806) 2024-04-24T22:46:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 0x55bf85ac65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f8886a00640 1 -- 172.21.15.111:0/3413374718 learned_addr learned my addr 172.21.15.111:0/3413374718 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 -- 172.21.15.111:0/3413374718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85ac8fe0 msgr2=0x55bf85ae0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85ac8fe0 0x55bf85ae0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 -- 172.21.15.111:0/3413374718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 msgr2=0x55bf85ac07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f887e000640 1 -- 172.21.15.111:0/3413374718 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf85ab8890 con 0x55bf85abe720 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.567+0000 7f8886a00640 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f887e000640 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 0x55bf85ac65a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88700ed180 tx=0x7f88700f4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d1d6208d3be899 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f8877400640 1 -- 172.21.15.111:0/3413374718 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870049040 con 0x55bf85abe720 2024-04-24T22:46:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f8877400640 1 -- 172.21.15.111:0/3413374718 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88700efe50 con 0x55bf85abe720 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f8877400640 1 -- 172.21.15.111:0/3413374718 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700bc010 con 0x55bf85abe720 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f8890dcc740 1 -- 172.21.15.111:0/3413374718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 msgr2=0x55bf85ac65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.568+0000 7f8890dcc740 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 0x55bf85ac65a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88700ed180 tx=0x7f88700f4040 comp rx=0 tx=0).stop 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 -- 172.21.15.111:0/3413374718 shutdown_connections 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85ac8fe0 0x55bf85ae0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85abe720 0x55bf85ac65a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 --2- 172.21.15.111:0/3413374718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abde00 0x55bf85ac07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 -- 172.21.15.111:0/3413374718 >> 172.21.15.111:0/3413374718 conn(0x55bf85aba310 msgr2=0x55bf85ac5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 -- 172.21.15.111:0/3413374718 shutdown_connections 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 -- 172.21.15.111:0/3413374718 wait complete. 2024-04-24T22:46:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 -- start start 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.569+0000 7f8890dcc740 1 Processor -- start 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 0x55bf85acd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f88881adb90 con 0x55bf85abe720 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88881af000 con 0x55bf85abde00 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ae260 con 0x55bf85ac8fe0 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8890dcc740 10 client.0 _ll_register_callbacks cb 0x7f8880009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8887400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47820/0 (socket says 172.21.15.111:47820) 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 -- 172.21.15.111:0/1120832081 learned_addr learned my addr 172.21.15.111:0/1120832081 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8887400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:51696/0 (socket says 172.21.15.111:51696) 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8886a00640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 0x55bf85acd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 -- 172.21.15.111:0/1120832081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 msgr2=0x55bf85b3e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 -- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 msgr2=0x55bf85acd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 0x55bf85acd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f887e000640 1 -- 172.21.15.111:0/1120832081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf85b3ea30 con 0x55bf85abe720 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8887400640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.570+0000 7f8886a00640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 0x55bf85acd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f887e000640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f887003d040 tx=0x7f88700bbde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a546a6532b135 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8874c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700d9020 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f88700ed400 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88700efe50 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8870049070 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8890dcc740 10 client.15543 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf85b3fd40 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.571+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88700efe50 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f88700c47f0 con 0x55bf85abe720 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 10 client.15543 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 20 client.15543 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85b40460 0x55bf85b42880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 4 client.15543 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:46:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf85a29340 con 0x55bf85b40460 2024-04-24T22:46:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8886a00640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85b40460 0x55bf85b42880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8886a00640 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85b40460 0x55bf85b42880 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950399b0eab2648f server_cookie=7f7db995b0513261 in_seq=0 out_seq=0 2024-04-24T22:46:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.572+0000 7f8874c00640 10 client.15543 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.587+0000 7f8874c00640 1 -- 172.21.15.111:0/1120832081 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf85a29340 con 0x55bf85b40460 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.587+0000 7f8874c00640 10 client.15543 handle_command_reply: tid=0 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.587+0000 7f8890dcc740 1 client.15543 shutdown 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.587+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 msgr2=0x55bf85acd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.587+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f887003d040 tx=0x7f88700bbde0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 shutdown_connections 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55bf85b40460 0x55bf85b42880 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf85ac8fe0 0x55bf85b3e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf85abe720 0x55bf85acd6c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 --2- 172.21.15.111:0/1120832081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bf85abde00 0x55bf85acd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8874c00640 0 client.15543 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 >> 172.21.15.111:0/1120832081 conn(0x55bf85aba310 msgr2=0x55bf85ac5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 shutdown_connections 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 1 -- 172.21.15.111:0/1120832081 wait complete. 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.588+0000 7f8890dcc740 20 client.15543 trim_cache size 0 max 16384 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:46:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:46:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:46:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 msgr2=0x7f886c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 0x7f886c0a5260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8880005fd0 tx=0x7f8880009810 comp rx=0 tx=0).stop 2024-04-24T22:46:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 msgr2=0x7f888819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f88700042f0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 shutdown_connections 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f888810e030 0x7f888819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8888074360 0x7f888819a640 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f886c0a2da0 0x7f886c0a5260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 --2- 172.21.15.111:0/4145862112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8888073a40 0x7f888819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 >> 172.21.15.111:0/4145862112 conn(0x7f8888078bd0 msgr2=0x7f88880fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 shutdown_connections 2024-04-24T22:46:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:10.589+0000 7f888de00640 1 -- 172.21.15.111:0/4145862112 wait complete. 2024-04-24T22:46:11.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:11 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:11.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:11 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:11.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:11 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:12.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:12 smithi171 ceph-mon[32337]: pgmap v130: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-24T22:46:12.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:12 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:12.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:12 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:46:12.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:12 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:46:12.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:12 smithi186 ceph-mon[32753]: pgmap v130: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-24T22:46:12.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:12 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:12.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:12 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:46:12.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:12 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:46:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:12 smithi111 ceph-mon[27746]: pgmap v130: 97 pgs: 97 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-24T22:46:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:12 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:12 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:46:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:12 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:46:14.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:14 smithi171 ceph-mon[32337]: pgmap v131: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-04-24T22:46:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:14 smithi186 ceph-mon[32753]: pgmap v131: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-04-24T22:46:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:14 smithi111 ceph-mon[27746]: pgmap v131: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-04-24T22:46:16.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:16 smithi171 ceph-mon[32337]: pgmap v132: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:16.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:16 smithi186 ceph-mon[32753]: pgmap v132: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:16 smithi111 ceph-mon[27746]: pgmap v132: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:18.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:18 smithi171 ceph-mon[32337]: pgmap v133: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:18.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:18 smithi186 ceph-mon[32753]: pgmap v133: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:18.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:18 smithi111 ceph-mon[27746]: pgmap v133: 97 pgs: 97 active+clean; 1.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:20 smithi171 ceph-mon[32337]: pgmap v134: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:20 smithi186 ceph-mon[32753]: pgmap v134: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:20 smithi111 ceph-mon[27746]: pgmap v134: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:22.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:22 smithi171 ceph-mon[32337]: pgmap v135: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:22.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:22 smithi186 ceph-mon[32753]: pgmap v135: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:22.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:22 smithi111 ceph-mon[27746]: pgmap v135: 97 pgs: 97 active+clean; 1.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 -- 172.21.15.111:0/3098638828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2380076590 msgr2=0x7f2380112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 --2- 172.21.15.111:0/3098638828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2380076590 0x7f2380112080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2368009930 tx=0x7f236802a440 comp rx=0 tx=0).stop 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 -- 172.21.15.111:0/3098638828 shutdown_connections 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 --2- 172.21.15.111:0/3098638828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23801125c0 0x7f23801149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 --2- 172.21.15.111:0/3098638828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2380076590 0x7f2380112080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 --2- 172.21.15.111:0/3098638828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380075c70 0x7f2380076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 -- 172.21.15.111:0/3098638828 >> 172.21.15.111:0/3098638828 conn(0x7f23800ff430 msgr2=0x7f2380101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 -- 172.21.15.111:0/3098638828 shutdown_connections 2024-04-24T22:46:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.734+0000 7f2384a00640 1 -- 172.21.15.111:0/3098638828 wait complete. 2024-04-24T22:46:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.735+0000 7f2384a00640 1 Processor -- start 2024-04-24T22:46:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 -- start start 2024-04-24T22:46:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380076590 0x7f238019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23801125c0 0x7f23801a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2380118180 con 0x7f2380075c70 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2380118000 con 0x7f2380076590 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f2384a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2380118300 con 0x7f23801125c0 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f237de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f237de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60774/0 (socket says 172.21.15.111:60774) 2024-04-24T22:46:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f237de00640 1 -- 172.21.15.111:0/1726511177 learned_addr learned my addr 172.21.15.111:0/1726511177 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f237d400640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380076590 0x7f238019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.736+0000 7f237e800640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23801125c0 0x7f23801a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 -- 172.21.15.111:0/1726511177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23801125c0 msgr2=0x7f23801a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23801125c0 0x7f23801a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 -- 172.21.15.111:0/1726511177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380076590 msgr2=0x7f238019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380076590 0x7f238019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 -- 172.21.15.111:0/1726511177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23801a3650 con 0x7f2380075c70 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f237de00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f237000eb90 tx=0x7f237000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f2376a00640 1 -- 172.21.15.111:0/1726511177 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370004d20 con 0x7f2380075c70 2024-04-24T22:46:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f2376a00640 1 -- 172.21.15.111:0/1726511177 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237000ee90 con 0x7f2380075c70 2024-04-24T22:46:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a3940 con 0x7f2380075c70 2024-04-24T22:46:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.737+0000 7f2376a00640 1 -- 172.21.15.111:0/1726511177 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370010830 con 0x7f2380075c70 2024-04-24T22:46:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.738+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801ab1d0 con 0x7f2380075c70 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 Processor -- start 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 -- start start 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2376a00640 1 -- 172.21.15.111:0/1726511177 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2370005020 con 0x7f2380075c70 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a35720 0x556c79a3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 0x556c79a423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c79998ab0 con 0x556c79a34e00 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c79941640 con 0x556c79a3ffe0 2024-04-24T22:46:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f2387830740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556c79994a70 con 0x556c79a35720 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f237e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f237e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60784/0 (socket says 172.21.15.111:60784) 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.741+0000 7f237e800640 1 -- 172.21.15.111:0/155460623 learned_addr learned my addr 172.21.15.111:0/155460623 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237de00640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a35720 0x556c79a3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237d400640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 0x556c79a423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 -- 172.21.15.111:0/155460623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a35720 msgr2=0x556c79a3faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a35720 0x556c79a3faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 -- 172.21.15.111:0/155460623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 msgr2=0x556c79a423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 0x556c79a423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 -- 172.21.15.111:0/155460623 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c796ad210 con 0x556c79a34e00 2024-04-24T22:46:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237d400640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 0x556c79a423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f237e800640 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ba0 tx=0x7f2378011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54822ff08c2b4a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f236f000640 1 -- 172.21.15.111:0/155460623 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378004230 con 0x556c79a34e00 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.742+0000 7f236f000640 1 -- 172.21.15.111:0/155460623 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2378009e10 con 0x556c79a34e00 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f236f000640 1 -- 172.21.15.111:0/155460623 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378004c40 con 0x556c79a34e00 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2376a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 0x7f23640a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2376a00640 1 -- 172.21.15.111:0/1726511177 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f23700cfa60 con 0x7f2380075c70 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f237de00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 0x7f23640a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 -- 172.21.15.111:0/155460623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 msgr2=0x556c79a351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ba0 tx=0x7f2378011810 comp rx=0 tx=0).stop 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 -- 172.21.15.111:0/155460623 shutdown_connections 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a3ffe0 0x556c79a423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a35720 0x556c79a3faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 --2- 172.21.15.111:0/155460623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a351e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.743+0000 7f2387830740 1 -- 172.21.15.111:0/155460623 >> 172.21.15.111:0/155460623 conn(0x556c79a31310 msgr2=0x556c79a316f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.744+0000 7f2387830740 1 -- 172.21.15.111:0/155460623 shutdown_connections 2024-04-24T22:46:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.744+0000 7f2387830740 1 -- 172.21.15.111:0/155460623 wait complete. 2024-04-24T22:46:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.744+0000 7f237de00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 0x7f23640a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x556c79a363a0 tx=0x7f23700d4040 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.744+0000 7f2387830740 1 -- start start 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.744+0000 7f2387830740 1 Processor -- start 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a3d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a3ffe0 0x556c79a3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c79a42c10 con 0x556c79a34e00 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c79954990 con 0x556c79a35720 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556c79a42d90 con 0x556c79a3ffe0 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f2387830740 10 client.0 _ll_register_callbacks cb 0x556c79956950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f237e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f237e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60790/0 (socket says 172.21.15.111:60790) 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f237e800640 1 -- 172.21.15.111:0/1185585102 learned_addr learned my addr 172.21.15.111:0/1185585102 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f237d400640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a3d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.745+0000 7f237e800640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a3ffe0 0x556c79a3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 -- 172.21.15.111:0/1185585102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a3ffe0 msgr2=0x556c79a3d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a3ffe0 0x556c79a3d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 -- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 msgr2=0x556c79a3d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a3d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 -- 172.21.15.111:0/1185585102 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c79779620 con 0x556c79a34e00 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f237e800640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f23780047e0 tx=0x7f2378009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12648ceec560dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2363400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237802d060 con 0x556c79a34e00 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f23780266c0 con 0x556c79a34e00 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2378009e10 con 0x556c79a34e00 2024-04-24T22:46:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f237801e040 con 0x556c79a34e00 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2387830740 10 client.15555 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.746+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556c79a4ea80 con 0x556c79a34e00 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2378009e10 con 0x556c79a34e00 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2378018070 con 0x556c79a34e00 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 10 client.15555 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 20 client.15555 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79a4fe20 0x556c79a52210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 4 client.15555 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c799a5ff0 con 0x556c79a4fe20 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.747+0000 7f237d400640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79a4fe20 0x556c79a52210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.748+0000 7f237d400640 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79a4fe20 0x556c79a52210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f58d96786ef71df server_cookie=69a1126f75001753 in_seq=0 out_seq=0 2024-04-24T22:46:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.748+0000 7f2363400640 10 client.15555 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2363400640 1 -- 172.21.15.111:0/1185585102 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c799a5ff0 con 0x556c79a4fe20 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2363400640 10 client.15555 handle_command_reply: tid=0 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 client.15555 shutdown 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 msgr2=0x556c79a3cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f23780047e0 tx=0x7f2378009620 comp rx=0 tx=0).stop 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 shutdown_connections 2024-04-24T22:46:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79a4fe20 0x556c79a52210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a3ffe0 0x556c79a3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a3d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 --2- 172.21.15.111:0/1185585102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a34e00 0x556c79a3cf00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 >> 172.21.15.111:0/1185585102 conn(0x556c79a31310 msgr2=0x556c79a37040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2363400640 0 client.15555 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.749+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 shutdown_connections 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.750+0000 7f2387830740 1 -- 172.21.15.111:0/1185585102 wait complete. 2024-04-24T22:46:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.750+0000 7f2387830740 20 client.15555 trim_cache size 0 max 16384 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 Processor -- start 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 -- start start 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a57930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b62e0 con 0x556c79a3ffe0 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b3bb0 con 0x556c79a35720 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f2387830740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b1a00 con 0x556c79a34e00 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f237e800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f237e800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:46474/0 (socket says 172.21.15.111:46474) 2024-04-24T22:46:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f237e800640 1 -- 172.21.15.111:0/3119669689 learned_addr learned my addr 172.21.15.111:0/3119669689 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f237e800640 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.766+0000 7f237d400640 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a57930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 -- 172.21.15.111:0/3119669689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 msgr2=0x556c79a57930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a57930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 -- 172.21.15.111:0/3119669689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 msgr2=0x556c79a57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a57e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 -- 172.21.15.111:0/3119669689 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c79a2f890 con 0x556c79a34e00 2024-04-24T22:46:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f237e800640 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ba0 tx=0x7f2378001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a6458c63a6ffc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f236e600640 1 -- 172.21.15.111:0/3119669689 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237802d070 con 0x556c79a34e00 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f236e600640 1 -- 172.21.15.111:0/3119669689 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2378028570 con 0x556c79a34e00 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.767+0000 7f236e600640 1 -- 172.21.15.111:0/3119669689 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378026700 con 0x556c79a34e00 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 -- 172.21.15.111:0/3119669689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 msgr2=0x556c79a37280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ba0 tx=0x7f2378001e50 comp rx=0 tx=0).stop 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 -- 172.21.15.111:0/3119669689 shutdown_connections 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79a57930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 --2- 172.21.15.111:0/3119669689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79a37280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 -- 172.21.15.111:0/3119669689 >> 172.21.15.111:0/3119669689 conn(0x556c79a31310 msgr2=0x556c79a30480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 -- 172.21.15.111:0/3119669689 shutdown_connections 2024-04-24T22:46:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.768+0000 7f2387830740 1 -- 172.21.15.111:0/3119669689 wait complete. 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 -- start start 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 Processor -- start 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79aae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79aaea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b2070 con 0x556c79a3ffe0 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b34e0 con 0x556c79a35720 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b2740 con 0x556c79a34e00 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.769+0000 7f2387830740 10 client.0 _ll_register_callbacks cb 0x7f2378009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237e800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79aae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237e800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79aae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:46490/0 (socket says 172.21.15.111:46490) 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237e800640 1 -- 172.21.15.111:0/3262755117 learned_addr learned my addr 172.21.15.111:0/3262755117 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237e800640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79aaea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 -- 172.21.15.111:0/3262755117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 msgr2=0x556c79aae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79aae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 -- 172.21.15.111:0/3262755117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 msgr2=0x556c79a453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 -- 172.21.15.111:0/3262755117 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c79a45ab0 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f237d400640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79aaea60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f236802a960 tx=0x7f2368004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a7d626c71c3e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.770+0000 7f2363400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368035040 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2378028570 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f2368045dd0 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2368005040 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 10 client.25625 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556c79ab5d80 con 0x556c79a35720 2024-04-24T22:46:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2378028570 con 0x556c79a35720 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f23680046c0 con 0x556c79a35720 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 10 client.25625 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 20 client.25625 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79ab64b0 0x556c79ab88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 4 client.25625 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.771+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c799a0340 con 0x556c79ab64b0 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.772+0000 7f237e800640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79ab64b0 0x556c79ab88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.772+0000 7f237e800640 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79ab64b0 0x556c79ab88d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f044f845beff5ed4 server_cookie=13b9b63dbbf182d in_seq=0 out_seq=0 2024-04-24T22:46:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.772+0000 7f2363400640 10 client.25625 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.786+0000 7f2363400640 1 -- 172.21.15.111:0/3262755117 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c799a0340 con 0x556c79ab64b0 2024-04-24T22:46:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.786+0000 7f2363400640 10 client.25625 handle_command_reply: tid=0 2024-04-24T22:46:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.786+0000 7f2387830740 1 client.25625 shutdown 2024-04-24T22:46:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 msgr2=0x556c79aaea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79aaea60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f236802a960 tx=0x7f2368004250 comp rx=0 tx=0).stop 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 shutdown_connections 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x556c79ab64b0 0x556c79ab88d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c79a3ffe0 0x556c79a453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c79a35720 0x556c79aaea60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2363400640 0 client.25625 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 --2- 172.21.15.111:0/3262755117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556c79a34e00 0x556c79aae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 >> 172.21.15.111:0/3262755117 conn(0x556c79a31310 msgr2=0x556c79a3e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 shutdown_connections 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 1 -- 172.21.15.111:0/3262755117 wait complete. 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.787+0000 7f2387830740 20 client.25625 trim_cache size 0 max 16384 2024-04-24T22:46:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:46:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:46:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:46:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:46:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 msgr2=0x7f23640a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 0x7f23640a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x556c79a363a0 tx=0x7f23700d4040 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 msgr2=0x7f238019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f237000eb90 tx=0x7f237000c6a0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 shutdown_connections 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23640a2e20 0x7f23640a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23801125c0 0x7f23801a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2380076590 0x7f238019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 --2- 172.21.15.111:0/1726511177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2380075c70 0x7f238019e660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 >> 172.21.15.111:0/1726511177 conn(0x7f23800ff430 msgr2=0x7f2380113230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 shutdown_connections 2024-04-24T22:46:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:23.788+0000 7f2384a00640 1 -- 172.21.15.111:0/1726511177 wait complete. 2024-04-24T22:46:24.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:24 smithi171 ceph-mon[32337]: pgmap v136: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:24.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:24 smithi186 ceph-mon[32753]: pgmap v136: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:24.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:24 smithi111 ceph-mon[27746]: pgmap v136: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:26 smithi171 ceph-mon[32337]: pgmap v137: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:26.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:26 smithi186 ceph-mon[32753]: pgmap v137: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:26.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:26 smithi111 ceph-mon[27746]: pgmap v137: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:28.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:28 smithi171 ceph-mon[32337]: pgmap v138: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:28.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:28 smithi186 ceph-mon[32753]: pgmap v138: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:28.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:28 smithi111 ceph-mon[27746]: pgmap v138: 97 pgs: 97 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:127/600: open fd = 3 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:128/600: open fd = 3 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.119 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:129/600: open fd = 3 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:130/600: open fd = 3 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:131/600: open fd = 3 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.120 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:132/600: open fd = 3 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:133/600: open fd = 3 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:134/600: open fd = 3 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:135/600: open fd = 3 2024-04-24T22:46:29.121 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:136/600: open fd = 3 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:137/600: open fd = 3 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:138/600: open fd = 3 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.122 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:139/600: open fd = 3 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:140/600: open fd = 3 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:141/600: open fd = 3 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:142/600: open fd = 3 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.123 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:143/600: open fd = 3 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:144/600: open fd = 3 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:145/600: open fd = 3 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.124 INFO:tasks.workunit.client.0.smithi111.stdout:146/600: open fd = 3 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:147/600: open fd = 3 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:148/600: open fd = 3 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:149/600: open fd = 3 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.125 INFO:tasks.workunit.client.0.smithi111.stdout:150/600: open fd = 3 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:151/600: open fd = 3 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:152/600: open fd = 3 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:153/600: open fd = 3 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.126 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:154/600: open fd = 3 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:155/600: open fd = 3 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:156/600: open fd = 3 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:157/600: open fd = 3 2024-04-24T22:46:29.127 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:158/600: open fd = 3 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:159/600: open fd = 3 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:160/600: open fd = 3 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.128 INFO:tasks.workunit.client.0.smithi111.stdout:161/600: open fd = 3 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:162/600: open fd = 3 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:163/600: open fd = 3 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:164/600: open fd = 3 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.129 INFO:tasks.workunit.client.0.smithi111.stdout:165/600: open fd = 3 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:166/600: open fd = 3 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:167/600: open fd = 3 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:168/600: open fd = 3 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.130 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:169/600: open fd = 3 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:170/600: open fd = 3 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:171/600: open fd = 3 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:172/600: open fd = 3 2024-04-24T22:46:29.131 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:173/600: open fd = 3 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:174/600: open fd = 3 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:175/600: open fd = 3 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.132 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:176/600: open fd = 3 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:177/600: open fd = 3 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:178/600: open fd = 3 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:179/600: open fd = 3 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.133 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:180/600: open fd = 3 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:181/600: open fd = 3 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:182/600: open fd = 3 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.134 INFO:tasks.workunit.client.0.smithi111.stdout:183/600: open fd = 3 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:184/600: open fd = 3 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:185/600: open fd = 3 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:186/600: open fd = 3 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.135 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:187/600: open fd = 3 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:188/600: open fd = 3 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:29.136 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:46:30.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:30 smithi171 ceph-mon[32337]: pgmap v139: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:30.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:30 smithi186 ceph-mon[32753]: pgmap v139: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:30.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:30 smithi111 ceph-mon[27746]: pgmap v139: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:32.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:32 smithi171 ceph-mon[32337]: pgmap v140: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-24T22:46:32.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:32 smithi186 ceph-mon[32753]: pgmap v140: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-24T22:46:32.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:32 smithi111 ceph-mon[27746]: pgmap v140: 97 pgs: 97 active+clean; 2.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-24T22:46:34.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:34 smithi171 ceph-mon[32337]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:46:34.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:34 smithi186 ceph-mon[32753]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:46:34.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:34 smithi111 ceph-mon[27746]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:46:36.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:36 smithi171 ceph-mon[32337]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:36.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:36 smithi186 ceph-mon[32753]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:36.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:36 smithi111 ceph-mon[27746]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 -- 172.21.15.111:0/3922225775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5410e030 msgr2=0x7f4e54110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/3922225775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5410e030 0x7f4e54110420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ca0 tx=0x7f4e4c02a950 comp rx=0 tx=0).stop 2024-04-24T22:46:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 -- 172.21.15.111:0/3922225775 shutdown_connections 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/3922225775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5410e030 0x7f4e54110420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/3922225775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54074360 0x7f4e5410daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/3922225775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54073a40 0x7f4e54073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 -- 172.21.15.111:0/3922225775 >> 172.21.15.111:0/3922225775 conn(0x7f4e54078bd0 msgr2=0x7f4e54078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 -- 172.21.15.111:0/3922225775 shutdown_connections 2024-04-24T22:46:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.933+0000 7f4e5ba00640 1 -- 172.21.15.111:0/3922225775 wait complete. 2024-04-24T22:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.934+0000 7f4e5ba00640 1 Processor -- start 2024-04-24T22:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.934+0000 7f4e5ba00640 1 -- start start 2024-04-24T22:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.934+0000 7f4e5ba00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 0x7f4e5419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.934+0000 7f4e5ba00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 0x7f4e5419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.934+0000 7f4e5ba00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e5ba00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54113db0 con 0x7f4e54073a40 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e5ba00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54113c30 con 0x7f4e5410e030 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e5ba00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54113f30 con 0x7f4e54074360 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 0x7f4e5419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 0x7f4e5419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35888/0 (socket says 172.21.15.111:35888) 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 -- 172.21.15.111:0/2520550822 learned_addr learned my addr 172.21.15.111:0/2520550822 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 0x7f4e5419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55370/0 (socket says 172.21.15.111:55370) 2024-04-24T22:46:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 -- 172.21.15.111:0/2520550822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 msgr2=0x7f4e5419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 0x7f4e5419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 -- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 msgr2=0x7f4e5419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 0x7f4e5419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e59a00640 1 -- 172.21.15.111:0/2520550822 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5419f0b0 con 0x7f4e5410e030 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.935+0000 7f4e53e00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 0x7f4e5419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e59a00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009dc0 tx=0x7f4e4c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e51600640 1 -- 172.21.15.111:0/2520550822 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c034040 con 0x7f4e5410e030 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5419f340 con 0x7f4e5410e030 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e51600640 1 -- 172.21.15.111:0/2520550822 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c02f940 con 0x7f4e5410e030 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e51600640 1 -- 172.21.15.111:0/2520550822 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0045a0 con 0x7f4e5410e030 2024-04-24T22:46:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.936+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a6be0 con 0x7f4e5410e030 2024-04-24T22:46:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.939+0000 7f4e5e80a740 1 Processor -- start 2024-04-24T22:46:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.939+0000 7f4e51600640 1 -- 172.21.15.111:0/2520550822 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4e4c037020 con 0x7f4e5410e030 2024-04-24T22:46:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 -- start start 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa38003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560fa3756ab0 con 0x560fa37f2e00 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560fa36ff640 con 0x560fa37f3720 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560fa3752a70 con 0x560fa37fdfe0 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.940+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35902/0 (socket says 172.21.15.111:35902) 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 -- 172.21.15.111:0/3974538017 learned_addr learned my addr 172.21.15.111:0/3974538017 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e53e00640 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa38003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 -- 172.21.15.111:0/3974538017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 msgr2=0x560fa38003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa38003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 -- 172.21.15.111:0/3974538017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 msgr2=0x560fa37f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 -- 172.21.15.111:0/3974538017 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa346b210 con 0x560fa37f3720 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e53e00640 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e59000640 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009d20 tx=0x7f4e48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1954d9593dbffc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.941+0000 7f4e46000640 1 -- 172.21.15.111:0/3974538017 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4801e070 con 0x560fa37f3720 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e46000640 1 -- 172.21.15.111:0/3974538017 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48019b10 con 0x560fa37f3720 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e46000640 1 -- 172.21.15.111:0/3974538017 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800eb90 con 0x560fa37f3720 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e51600640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 0x7f4e3c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e51600640 1 -- 172.21.15.111:0/2520550822 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f4e4c0f0070 con 0x7f4e5410e030 2024-04-24T22:46:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e53e00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 0x7f4e3c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3974538017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 msgr2=0x560fa37fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.942+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009d20 tx=0x7f4e48011810 comp rx=0 tx=0).stop 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3974538017 shutdown_connections 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa38003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fdaa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3974538017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3974538017 >> 172.21.15.111:0/3974538017 conn(0x560fa37ef310 msgr2=0x560fa37ef6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3974538017 shutdown_connections 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3974538017 wait complete. 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e53e00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 0x7f4e3c0a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x560fa37f0730 tx=0x7f4e40009cc0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 -- start start 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.943+0000 7f4e5e80a740 1 Processor -- start 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f3720 0x560fa37fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37fdfe0 0x560fa37fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560fa3800c10 con 0x560fa37f2e00 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560fa3712990 con 0x560fa37fdfe0 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560fa3800d90 con 0x560fa37f3720 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e5e80a740 10 client.0 _ll_register_callbacks cb 0x560fa3714950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55396/0 (socket says 172.21.15.111:55396) 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 -- 172.21.15.111:0/1813398736 learned_addr learned my addr 172.21.15.111:0/1813398736 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e53e00640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f3720 0x560fa37fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 -- 172.21.15.111:0/1813398736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f3720 msgr2=0x560fa37fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f3720 0x560fa37fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 -- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37fdfe0 msgr2=0x560fa37fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37fdfe0 0x560fa37fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.944+0000 7f4e59000640 1 -- 172.21.15.111:0/1813398736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa3537620 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e59000640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011d10 tx=0x7f4e48019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41085d1f941352e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48027020 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e48019b10 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f4e4800eaf0 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4e48032050 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e5e80a740 10 client.15573 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.945+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560fa3815320 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48019b10 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4e480046c0 con 0x560fa37f2e00 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 10 client.15573 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 20 client.15573 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa3816730 0x560fa3818b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 4 client.15573 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.946+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa3763ff0 con 0x560fa3816730 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.948+0000 7f4e53e00640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa3816730 0x560fa3818b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.951+0000 7f4e53e00640 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa3816730 0x560fa3818b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80e0b57c3dfc75d server_cookie=6d58612dd54859b4 in_seq=0 out_seq=0 2024-04-24T22:46:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.951+0000 7f4e3aa00640 10 client.15573 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e3aa00640 1 -- 172.21.15.111:0/1813398736 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fa3763ff0 con 0x560fa3816730 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e3aa00640 10 client.15573 handle_command_reply: tid=0 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 client.15573 shutdown 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 msgr2=0x560fa37faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011d10 tx=0x7f4e48019660 comp rx=0 tx=0).stop 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 shutdown_connections 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa3816730 0x560fa3818b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37fdfe0 0x560fa37fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f3720 0x560fa37fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1813398736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa37faf40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e3aa00640 0 client.15573 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 >> 172.21.15.111:0/1813398736 conn(0x560fa37ef310 msgr2=0x560fa37f5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 shutdown_connections 2024-04-24T22:46:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1813398736 wait complete. 2024-04-24T22:46:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.952+0000 7f4e5e80a740 20 client.15573 trim_cache size 0 max 16384 2024-04-24T22:46:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.968+0000 7f4e5e80a740 1 Processor -- start 2024-04-24T22:46:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.968+0000 7f4e5e80a740 1 -- start start 2024-04-24T22:46:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 0x560fa3812390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 0x560fa381e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541ad960 con 0x560fa37fdfe0 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541ab230 con 0x560fa37f3720 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a90c0 con 0x560fa37f2e00 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e59000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 0x560fa3812390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35912/0 (socket says 172.21.15.111:35912) 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e53e00640 1 -- 172.21.15.111:0/1298951632 learned_addr learned my addr 172.21.15.111:0/1298951632 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.969+0000 7f4e59a00640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 0x560fa381e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 -- 172.21.15.111:0/1298951632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 msgr2=0x560fa37fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 -- 172.21.15.111:0/1298951632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 msgr2=0x560fa381e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 0x560fa381e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 -- 172.21.15.111:0/1298951632 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa3800600 con 0x560fa37f2e00 2024-04-24T22:46:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e53e00640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59a00640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 0x560fa381e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e59000640 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 0x560fa3812390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011850 tx=0x7f4e480096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94dd9912ed655cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e45600640 1 -- 172.21.15.111:0/1298951632 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015020 con 0x560fa37f2e00 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e45600640 1 -- 172.21.15.111:0/1298951632 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4801f040 con 0x560fa37f2e00 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.970+0000 7f4e45600640 1 -- 172.21.15.111:0/1298951632 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48027040 con 0x560fa37f2e00 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1298951632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 msgr2=0x560fa3812390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 0x560fa3812390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011850 tx=0x7f4e480096a0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1298951632 shutdown_connections 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37fdfe0 0x560fa381e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa37fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/1298951632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37f2e00 0x560fa3812390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1298951632 >> 172.21.15.111:0/1298951632 conn(0x560fa37ef310 msgr2=0x560fa37faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1298951632 shutdown_connections 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- 172.21.15.111:0/1298951632 wait complete. 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.971+0000 7f4e5e80a740 1 -- start start 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 Processor -- start 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa380e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa380b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a9710 con 0x560fa37f2e00 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541aab60 con 0x560fa37f3720 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a9dc0 con 0x560fa37fdfe0 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e5e80a740 10 client.0 _ll_register_callbacks cb 0x7f4e4801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.972+0000 7f4e59000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa380e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35924/0 (socket says 172.21.15.111:35924) 2024-04-24T22:46:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 -- 172.21.15.111:0/3559494758 learned_addr learned my addr 172.21.15.111:0/3559494758 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e59a00640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa380b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 -- 172.21.15.111:0/3559494758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 msgr2=0x560fa380b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa380b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 -- 172.21.15.111:0/3559494758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 msgr2=0x560fa380e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa380e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 -- 172.21.15.111:0/3559494758 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa380be30 con 0x560fa37f3720 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e59a00640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa380b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e59000640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa380e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e53e00640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x560fa37f04a0 tx=0x7f4e40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335df8ff6f957058 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.973+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400035f0 con 0x560fa37f3720 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e4801f040 con 0x560fa37f3720 2024-04-24T22:46:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f4e40003790 con 0x560fa37f3720 2024-04-24T22:46:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4e40003990 con 0x560fa37f3720 2024-04-24T22:46:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 10 client.25640 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560fa387d4b0 con 0x560fa37f3720 2024-04-24T22:46:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4801f040 con 0x560fa37f3720 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4e40012b00 con 0x560fa37f3720 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 10 client.25640 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 20 client.25640 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa387dc30 0x560fa3880050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 4 client.25640 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.974+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fa375e340 con 0x560fa387dc30 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.975+0000 7f4e59000640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa387dc30 0x560fa3880050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.975+0000 7f4e59000640 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa387dc30 0x560fa3880050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9c34e2e8392c52 server_cookie=d81a90440f836a9 in_seq=0 out_seq=0 2024-04-24T22:46:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.975+0000 7f4e3aa00640 10 client.25640 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e3aa00640 1 -- 172.21.15.111:0/3559494758 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fa375e340 con 0x560fa387dc30 2024-04-24T22:46:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e3aa00640 10 client.25640 handle_command_reply: tid=0 2024-04-24T22:46:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 client.25640 shutdown 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 msgr2=0x560fa380ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x560fa37f04a0 tx=0x7f4e40004730 comp rx=0 tx=0).stop 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 shutdown_connections 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560fa387dc30 0x560fa3880050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560fa37fdfe0 0x560fa380b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560fa37f3720 0x560fa380ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 --2- 172.21.15.111:0/3559494758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa37f2e00 0x560fa380e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e3aa00640 0 client.25640 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 >> 172.21.15.111:0/3559494758 conn(0x560fa37ef310 msgr2=0x560fa37fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 shutdown_connections 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.989+0000 7f4e5e80a740 1 -- 172.21.15.111:0/3559494758 wait complete. 2024-04-24T22:46:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.990+0000 7f4e5e80a740 20 client.25640 trim_cache size 0 max 16384 2024-04-24T22:46:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:46:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:46:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:46:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.990+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 msgr2=0x7f4e3c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.990+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 0x7f4e3c0a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x560fa37f0730 tx=0x7f4e40009cc0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.990+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 msgr2=0x7f4e5419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.990+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009dc0 tx=0x7f4e4c003ba0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 shutdown_connections 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e5410e030 0x7f4e5419e930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4e3c0a2da0 0x7f4e3c0a5260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54074360 0x7f4e5419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 --2- 172.21.15.111:0/2520550822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e54073a40 0x7f4e5419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 >> 172.21.15.111:0/2520550822 conn(0x7f4e54078bd0 msgr2=0x7f4e540fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 shutdown_connections 2024-04-24T22:46:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:36.991+0000 7f4e5ba00640 1 -- 172.21.15.111:0/2520550822 wait complete. 2024-04-24T22:46:38.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:38 smithi171 ceph-mon[32337]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:38.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:38 smithi186 ceph-mon[32753]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:38 smithi111 ceph-mon[27746]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:40.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:40 smithi171 ceph-mon[32337]: pgmap v144: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-24T22:46:40.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:40 smithi186 ceph-mon[32753]: pgmap v144: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-24T22:46:40.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:40 smithi111 ceph-mon[27746]: pgmap v144: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-24T22:46:42.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:42 smithi171 ceph-mon[32337]: pgmap v145: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:42.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:42 smithi186 ceph-mon[32753]: pgmap v145: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:42.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:42 smithi111 ceph-mon[27746]: pgmap v145: 97 pgs: 97 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:44.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:44 smithi171 ceph-mon[32337]: pgmap v146: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:44 smithi186 ceph-mon[32753]: pgmap v146: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:44.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:44 smithi111 ceph-mon[27746]: pgmap v146: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:46.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:46 smithi186 ceph-mon[32753]: pgmap v147: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:46.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:46 smithi111 ceph-mon[27746]: pgmap v147: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:46.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:46 smithi171 ceph-mon[32337]: pgmap v147: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:48 smithi186 ceph-mon[32753]: pgmap v148: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:48 smithi111 ceph-mon[27746]: pgmap v148: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:48 smithi171 ceph-mon[32337]: pgmap v148: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-24T22:46:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.132+0000 7fcb83e00640 1 -- 172.21.15.111:0/3186773842 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb780319b0 con 0x7fcb84109790 2024-04-24T22:46:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 -- 172.21.15.111:0/3186773842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 msgr2=0x7fcb84109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/3186773842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb84109b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009930 tx=0x7fcb7802a420 comp rx=0 tx=0).stop 2024-04-24T22:46:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 -- 172.21.15.111:0/3186773842 shutdown_connections 2024-04-24T22:46:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/3186773842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb84110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/3186773842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 0x7fcb84103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/3186773842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb84109b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 -- 172.21.15.111:0/3186773842 >> 172.21.15.111:0/3186773842 conn(0x7fcb840ff250 msgr2=0x7fcb84101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 -- 172.21.15.111:0/3186773842 shutdown_connections 2024-04-24T22:46:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.133+0000 7fcb8ae00640 1 -- 172.21.15.111:0/3186773842 wait complete. 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 Processor -- start 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 -- start start 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 0x7fcb8419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb8419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113ba0 con 0x7fcb84109790 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113a20 con 0x7fcb84103780 2024-04-24T22:46:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.134+0000 7fcb8ae00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113d20 con 0x7fcb84104140 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb8419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50440/0 (socket says 172.21.15.111:50440) 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 -- 172.21.15.111:0/427918410 learned_addr learned my addr 172.21.15.111:0/427918410 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 0x7fcb8419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 -- 172.21.15.111:0/427918410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 msgr2=0x7fcb8419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 0x7fcb8419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 -- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 msgr2=0x7fcb8419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb8419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 -- 172.21.15.111:0/427918410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8419f080 con 0x7fcb84104140 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.135+0000 7fcb89000640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d950 tx=0x7fcb7000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.136+0000 7fcb82000640 1 -- 172.21.15.111:0/427918410 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70017030 con 0x7fcb84104140 2024-04-24T22:46:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.136+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8419f370 con 0x7fcb84104140 2024-04-24T22:46:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.136+0000 7fcb82000640 1 -- 172.21.15.111:0/427918410 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb70004020 con 0x7fcb84104140 2024-04-24T22:46:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.136+0000 7fcb82000640 1 -- 172.21.15.111:0/427918410 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb700047a0 con 0x7fcb84104140 2024-04-24T22:46:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.136+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a6c00 con 0x7fcb84104140 2024-04-24T22:46:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.139+0000 7fcb82000640 1 -- 172.21.15.111:0/427918410 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcb700041d0 con 0x7fcb84104140 2024-04-24T22:46:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.139+0000 7fcb8f17a740 1 Processor -- start 2024-04-24T22:46:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.139+0000 7fcb8f17a740 1 -- start start 2024-04-24T22:46:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616bf8d0 0x5642616c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564261622ab0 con 0x5642616ca190 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5642615cb640 con 0x5642616befb0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56426161ea70 con 0x5642616bf8d0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8a400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8a400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35440/0 (socket says 172.21.15.111:35440) 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb8a400640 1 -- 172.21.15.111:0/3564751954 learned_addr learned my addr 172.21.15.111:0/3564751954 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.140+0000 7fcb89a00640 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616bf8d0 0x5642616c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 -- 172.21.15.111:0/3564751954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616bf8d0 msgr2=0x5642616c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616bf8d0 0x5642616c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 -- 172.21.15.111:0/3564751954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 msgr2=0x5642616cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 -- 172.21.15.111:0/3564751954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564261337210 con 0x5642616befb0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb8a400640 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009f70 tx=0x7fcb7c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8060bbc49d9f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb82000640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 0x7fcb6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb82000640 1 -- 172.21.15.111:0/427918410 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcb700cedb0 con 0x7fcb84104140 2024-04-24T22:46:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb76a00640 1 -- 172.21.15.111:0/3564751954 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c016070 con 0x5642616befb0 2024-04-24T22:46:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb89a00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 0x7fcb6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb76a00640 1 -- 172.21.15.111:0/3564751954 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7c004d20 con 0x5642616befb0 2024-04-24T22:46:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.141+0000 7fcb76a00640 1 -- 172.21.15.111:0/3564751954 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c012870 con 0x5642616befb0 2024-04-24T22:46:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 -- 172.21.15.111:0/3564751954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 msgr2=0x5642616bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009f70 tx=0x7fcb7c010740 comp rx=0 tx=0).stop 2024-04-24T22:46:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb89a00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 0x7fcb6c0a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009900 tx=0x7fcb780058d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 -- 172.21.15.111:0/3564751954 shutdown_connections 2024-04-24T22:46:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616bf8d0 0x5642616c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/3564751954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616bf390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 -- 172.21.15.111:0/3564751954 >> 172.21.15.111:0/3564751954 conn(0x5642616bb4c0 msgr2=0x5642616bb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 -- 172.21.15.111:0/3564751954 shutdown_connections 2024-04-24T22:46:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.142+0000 7fcb8f17a740 1 -- 172.21.15.111:0/3564751954 wait complete. 2024-04-24T22:46:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.143+0000 7fcb8f17a740 1 -- start start 2024-04-24T22:46:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.143+0000 7fcb8f17a740 1 Processor -- start 2024-04-24T22:46:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.143+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616bf8d0 0x5642616c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5642616ccdc0 con 0x5642616ca190 2024-04-24T22:46:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5642615de990 con 0x5642616bf8d0 2024-04-24T22:46:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5642616ccf40 con 0x5642616befb0 2024-04-24T22:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8f17a740 10 client.0 _ll_register_callbacks cb 0x5642615e0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50458/0 (socket says 172.21.15.111:50458) 2024-04-24T22:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 -- 172.21.15.111:0/1032356163 learned_addr learned my addr 172.21.15.111:0/1032356163 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8a400640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89000640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616bf8d0 0x5642616c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 -- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616bf8d0 msgr2=0x5642616c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616bf8d0 0x5642616c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 -- 172.21.15.111:0/1032356163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 msgr2=0x5642616c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb89a00640 1 -- 172.21.15.111:0/1032356163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564261403620 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.144+0000 7fcb8a400640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb89a00640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009930 tx=0x7fcb78009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d27c9742999359 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb6b400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7804e070 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcb78003cc0 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7c004d20 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcb7805a050 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb8f17a740 10 client.25675 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.145+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5642616d8ce0 con 0x5642616befb0 2024-04-24T22:46:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7c004d20 con 0x5642616befb0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcb7803d4e0 con 0x5642616befb0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 10 client.25675 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 20 client.25675 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642616da1a0 0x5642616dc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 4 client.25675 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.146+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56426162fff0 con 0x5642616da1a0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.147+0000 7fcb89000640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642616da1a0 0x5642616dc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.147+0000 7fcb89000640 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642616da1a0 0x5642616dc590 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25f839c0ff836f6 server_cookie=dce36b29b996eb35 in_seq=0 out_seq=0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.147+0000 7fcb6b400640 10 client.25675 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb6b400640 1 -- 172.21.15.111:0/1032356163 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56426162fff0 con 0x5642616da1a0 2024-04-24T22:46:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb6b400640 10 client.25675 handle_command_reply: tid=0 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 client.25675 shutdown 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 msgr2=0x5642616c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009930 tx=0x7fcb78009260 comp rx=0 tx=0).stop 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 shutdown_connections 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642616da1a0 0x5642616dc590 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616ca190 0x5642616c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616bf8d0 0x5642616c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1032356163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616befb0 0x5642616c7160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb6b400640 0 client.25675 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.148+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 >> 172.21.15.111:0/1032356163 conn(0x5642616bb4c0 msgr2=0x5642616c11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.149+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 shutdown_connections 2024-04-24T22:46:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.149+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1032356163 wait complete. 2024-04-24T22:46:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.149+0000 7fcb8f17a740 20 client.25675 trim_cache size 0 max 16384 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 Processor -- start 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 -- start start 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x5642616e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841b1ce0 con 0x5642616bf8d0 2024-04-24T22:46:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841af5b0 con 0x5642616befb0 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ad400 con 0x5642616ca190 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb8a400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x5642616e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54536/0 (socket says 172.21.15.111:54536) 2024-04-24T22:46:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35464/0 (socket says 172.21.15.111:35464) 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.165+0000 7fcb89000640 1 -- 172.21.15.111:0/1401088344 learned_addr learned my addr 172.21.15.111:0/1401088344 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 -- 172.21.15.111:0/1401088344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 msgr2=0x5642616e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x5642616e23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 -- 172.21.15.111:0/1401088344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 msgr2=0x5642616c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 -- 172.21.15.111:0/1401088344 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642616cc8f0 con 0x5642616bf8d0 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb8a400640 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x5642616e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89a00640 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:46:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.166+0000 7fcb89000640 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcb7009afa0 tx=0x7fcb700d1200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0b3d45859b9c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.170+0000 7fcb76000640 1 -- 172.21.15.111:0/1401088344 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7009b070 con 0x5642616bf8d0 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.170+0000 7fcb76000640 1 -- 172.21.15.111:0/1401088344 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7009b9a0 con 0x5642616bf8d0 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.170+0000 7fcb76000640 1 -- 172.21.15.111:0/1401088344 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7009ac00 con 0x5642616bf8d0 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1401088344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 msgr2=0x5642616e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcb7009afa0 tx=0x7fcb700d1200 comp rx=0 tx=0).stop 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1401088344 shutdown_connections 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x5642616e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616e1e60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/1401088344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1401088344 >> 172.21.15.111:0/1401088344 conn(0x5642616bb4c0 msgr2=0x5642616c0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1401088344 shutdown_connections 2024-04-24T22:46:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- 172.21.15.111:0/1401088344 wait complete. 2024-04-24T22:46:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 -- start start 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.171+0000 7fcb8f17a740 1 Processor -- start 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x56426173f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ada70 con 0x5642616bf8d0 2024-04-24T22:46:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841aeee0 con 0x5642616befb0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ae140 con 0x5642616ca190 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8f17a740 10 client.0 _ll_register_callbacks cb 0x7fcb78005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb8a400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x56426173f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb89a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35468/0 (socket says 172.21.15.111:35468) 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.172+0000 7fcb89a00640 1 -- 172.21.15.111:0/2028362080 learned_addr learned my addr 172.21.15.111:0/2028362080 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 -- 172.21.15.111:0/2028362080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 msgr2=0x56426173f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x56426173f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 -- 172.21.15.111:0/2028362080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 msgr2=0x5642616ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 -- 172.21.15.111:0/2028362080 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426173fc10 con 0x5642616befb0 2024-04-24T22:46:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb89a00640 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb7803daa0 tx=0x7fcb78009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c31ab5e3113feb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb6b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7804e040 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fcb780039c0 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.173+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7009b9a0 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcb7806d050 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 10 client.25658 fetch_fsmap learned FSMap version 24 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564261740e80 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7009b9a0 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcb7804a9c0 con 0x5642616befb0 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 10 client.25658 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 20 client.25658 populate_metadata read hostname 'smithi111' 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642617415f0 0x564261743a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 4 client.25658 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:46:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.174+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56426162a340 con 0x5642617415f0 2024-04-24T22:46:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.175+0000 7fcb89000640 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642617415f0 0x564261743a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.175+0000 7fcb89000640 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642617415f0 0x564261743a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158258796af41756 server_cookie=ae49f6a1ba98af26 in_seq=0 out_seq=0 2024-04-24T22:46:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.175+0000 7fcb6b400640 10 client.25658 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.189+0000 7fcb6b400640 1 -- 172.21.15.111:0/2028362080 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56426162a340 con 0x5642617415f0 2024-04-24T22:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.189+0000 7fcb6b400640 10 client.25658 handle_command_reply: tid=0 2024-04-24T22:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.189+0000 7fcb8f17a740 1 client.25658 shutdown 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 msgr2=0x5642616ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb7803daa0 tx=0x7fcb78009680 comp rx=0 tx=0).stop 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 shutdown_connections 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5642617415f0 0x564261743a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642616ca190 0x56426173f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642616bf8d0 0x5642616ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 --2- 172.21.15.111:0/2028362080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642616befb0 0x5642616ce230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb6b400640 0 client.25658 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 >> 172.21.15.111:0/2028362080 conn(0x5642616bb4c0 msgr2=0x5642616c0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 shutdown_connections 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 1 -- 172.21.15.111:0/2028362080 wait complete. 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.190+0000 7fcb8f17a740 20 client.25658 trim_cache size 0 max 16384 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 msgr2=0x7fcb6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 0x7fcb6c0a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009900 tx=0x7fcb780058d0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 msgr2=0x7fcb8419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d950 tx=0x7fcb7000de10 comp rx=0 tx=0).stop 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 shutdown_connections 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb84109790 0x7fcb8419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcb6c0a2cf0 0x7fcb6c0a51b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84104140 0x7fcb8419a620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 --2- 172.21.15.111:0/427918410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84103780 0x7fcb8419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 >> 172.21.15.111:0/427918410 conn(0x7fcb840ff250 msgr2=0x7fcb840ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 shutdown_connections 2024-04-24T22:46:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:46:50.191+0000 7fcb8ae00640 1 -- 172.21.15.111:0/427918410 wait complete. 2024-04-24T22:46:50.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:50 smithi186 ceph-mon[32753]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:50.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:50 smithi111 ceph-mon[27746]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:50.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:50 smithi171 ceph-mon[32337]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-24T22:46:52.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:52 smithi186 ceph-mon[32753]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:52.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:52 smithi111 ceph-mon[27746]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:52.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:52 smithi171 ceph-mon[32337]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-04-24T22:46:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:54 smithi186 ceph-mon[32753]: pgmap v151: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:54.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:54 smithi111 ceph-mon[27746]: pgmap v151: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:54.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:54 smithi171 ceph-mon[32337]: pgmap v151: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-24T22:46:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:56 smithi186 ceph-mon[32753]: pgmap v152: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:56.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:56 smithi111 ceph-mon[27746]: pgmap v152: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:56 smithi171 ceph-mon[32337]: pgmap v152: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:58.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:46:58 smithi186 ceph-mon[32753]: pgmap v153: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:58.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:46:58 smithi111 ceph-mon[27746]: pgmap v153: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:46:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:46:58 smithi171 ceph-mon[32337]: pgmap v153: 97 pgs: 97 active+clean; 3.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:47:00.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:00 smithi186 ceph-mon[32753]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:47:00.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:00 smithi111 ceph-mon[27746]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:47:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:00 smithi171 ceph-mon[32337]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-04-24T22:47:02.459 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:02 smithi171 ceph-mon[32337]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:47:02.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:02 smithi186 ceph-mon[32753]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:47:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:02 smithi111 ceph-mon[27746]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-24T22:47:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 -- 172.21.15.111:0/4216360501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 msgr2=0x7f64f810dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 --2- 172.21.15.111:0/4216360501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f810dff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009930 tx=0x7f64e802a420 comp rx=0 tx=0).stop 2024-04-24T22:47:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 -- 172.21.15.111:0/4216360501 shutdown_connections 2024-04-24T22:47:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 --2- 172.21.15.111:0/4216360501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 0x7f64f8114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 --2- 172.21.15.111:0/4216360501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 --2- 172.21.15.111:0/4216360501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f810dff0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 -- 172.21.15.111:0/4216360501 >> 172.21.15.111:0/4216360501 conn(0x7f64f8077520 msgr2=0x7f64f8106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 -- 172.21.15.111:0/4216360501 shutdown_connections 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.332+0000 7f64f7200640 1 -- 172.21.15.111:0/4216360501 wait complete. 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.333+0000 7f64f7200640 1 Processor -- start 2024-04-24T22:47:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 -- start start 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 0x7f64f819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f81a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8117fd0 con 0x7f64f8108600 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8117e50 con 0x7f64f8107c80 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f7200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8118150 con 0x7f64f810dc10 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:59966/0 (socket says 172.21.15.111:59966) 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64f5e00640 1 -- 172.21.15.111:0/1725877075 learned_addr learned my addr 172.21.15.111:0/1725877075 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.334+0000 7f64efe00640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 0x7f64f819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f6800640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f81a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 -- 172.21.15.111:0/1725877075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 msgr2=0x7f64f81a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f81a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 -- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 msgr2=0x7f64f819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 0x7f64f819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 -- 172.21.15.111:0/1725877075 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81a3480 con 0x7f64f8107c80 2024-04-24T22:47:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f64e802a940 tx=0x7f64e8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.335+0000 7f64eea00640 1 -- 172.21.15.111:0/1725877075 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e803f070 con 0x7f64f8107c80 2024-04-24T22:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.336+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f81a3710 con 0x7f64f8107c80 2024-04-24T22:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.336+0000 7f64eea00640 1 -- 172.21.15.111:0/1725877075 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8031c90 con 0x7f64f8107c80 2024-04-24T22:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.336+0000 7f64eea00640 1 -- 172.21.15.111:0/1725877075 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8035030 con 0x7f64f8107c80 2024-04-24T22:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.336+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f81ab000 con 0x7f64f8107c80 2024-04-24T22:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.339+0000 7f64eea00640 1 -- 172.21.15.111:0/1725877075 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f64e8042070 con 0x7f64f8107c80 2024-04-24T22:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.339+0000 7f64ffd94740 1 Processor -- start 2024-04-24T22:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.339+0000 7f64ffd94740 1 -- start start 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4aac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac403ab0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac3ac640 con 0x55d0ac49ffb0 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64ffd94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac3ffa70 con 0x55d0ac4ab190 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:59972/0 (socket says 172.21.15.111:59972) 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 -- 172.21.15.111:0/1038739294 learned_addr learned my addr 172.21.15.111:0/1038739294 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64efe00640 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64eea00640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 -- 172.21.15.111:0/1038739294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 msgr2=0x55d0ac4ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 -- 172.21.15.111:0/1038739294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 msgr2=0x55d0ac4aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.340+0000 7f64f6800640 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.341+0000 7f64f6800640 1 -- 172.21.15.111:0/1038739294 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ac118210 con 0x55d0ac49ffb0 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.341+0000 7f64eea00640 1 -- 172.21.15.111:0/1725877075 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f64e80efd60 con 0x7f64f8107c80 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.341+0000 7f64f6800640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.341+0000 7f64f6800640 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009f60 tx=0x7f64f0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=425146874296c2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64e7400640 1 -- 172.21.15.111:0/1038739294 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f000f680 con 0x55d0ac49ffb0 2024-04-24T22:47:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64e7400640 1 -- 172.21.15.111:0/1038739294 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f000fd00 con 0x55d0ac49ffb0 2024-04-24T22:47:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64e7400640 1 -- 172.21.15.111:0/1038739294 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f001bbf0 con 0x55d0ac49ffb0 2024-04-24T22:47:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64f6800640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 0x7f64dc0a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64f0020000 tx=0x7f64f0021000 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 -- 172.21.15.111:0/1038739294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 msgr2=0x55d0ac4a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009f60 tx=0x7f64f0009f90 comp rx=0 tx=0).stop 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 -- 172.21.15.111:0/1038739294 shutdown_connections 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1038739294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a0390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.342+0000 7f64ffd94740 1 -- 172.21.15.111:0/1038739294 >> 172.21.15.111:0/1038739294 conn(0x55d0ac49c4c0 msgr2=0x55d0ac49c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.343+0000 7f64ffd94740 1 -- 172.21.15.111:0/1038739294 shutdown_connections 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.343+0000 7f64ffd94740 1 -- 172.21.15.111:0/1038739294 wait complete. 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.343+0000 7f64ffd94740 1 -- start start 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.343+0000 7f64ffd94740 1 Processor -- start 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac4addc0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac3bf990 con 0x55d0ac4ab190 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac4adf40 con 0x55d0ac49ffb0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64ffd94740 10 client.0 _ll_register_callbacks cb 0x55d0ac3c1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33264/0 (socket says 172.21.15.111:33264) 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 -- 172.21.15.111:0/2403170462 learned_addr learned my addr 172.21.15.111:0/2403170462 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 -- 172.21.15.111:0/2403170462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 msgr2=0x55d0ac4a8090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 -- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 msgr2=0x55d0ac4a8b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a8b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.344+0000 7f64efe00640 1 -- 172.21.15.111:0/2403170462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ac1e4620 con 0x55d0ac4a08d0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64efe00640 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f64e0012700 tx=0x7f64e0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd78ed17664b02f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:47:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004020 con 0x55d0ac4a08d0 2024-04-24T22:47:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f64e00041c0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f000fd00 con 0x55d0ac4a08d0 2024-04-24T22:47:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f64e00043c0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64ffd94740 10 client.15609 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.345+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d0ac4b9c90 con 0x55d0ac4a08d0 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f000fd00 con 0x55d0ac4a08d0 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f64e0003aa0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 20 client.15609 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac4bb150 0x55d0ac4bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 4 client.15609 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.346+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0ac410ff0 con 0x55d0ac4bb150 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.347+0000 7f64f5e00640 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac4bb150 0x55d0ac4bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.347+0000 7f64f5e00640 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac4bb150 0x55d0ac4bd560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ef92ccef5fae69 server_cookie=d269565939a62ab1 in_seq=0 out_seq=0 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.347+0000 7f64dbe00640 10 client.15609 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64dbe00640 1 -- 172.21.15.111:0/2403170462 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0ac410ff0 con 0x55d0ac4bb150 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64dbe00640 10 client.15609 handle_command_reply: tid=0 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 client.15609 shutdown 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 msgr2=0x55d0ac4a85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f64e0012700 tx=0x7f64e0012bc0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 shutdown_connections 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac4bb150 0x55d0ac4bd560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4a85d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 --2- 172.21.15.111:0/2403170462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64dbe00640 0 client.15609 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 >> 172.21.15.111:0/2403170462 conn(0x55d0ac49c4c0 msgr2=0x55d0ac4a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 shutdown_connections 2024-04-24T22:47:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.348+0000 7f64ffd94740 1 -- 172.21.15.111:0/2403170462 wait complete. 2024-04-24T22:47:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.349+0000 7f64ffd94740 20 client.15609 trim_cache size 0 max 16384 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 Processor -- start 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 -- start start 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 0x55d0ac49bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b6130 con 0x55d0ac49ffb0 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b3a00 con 0x55d0ac4a08d0 2024-04-24T22:47:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64ffd94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b1850 con 0x55d0ac4ab190 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33270/0 (socket says 172.21.15.111:33270) 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.365+0000 7f64f5e00640 1 -- 172.21.15.111:0/1397200858 learned_addr learned my addr 172.21.15.111:0/1397200858 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f6800640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64efe00640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 0x55d0ac49bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 -- 172.21.15.111:0/1397200858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 msgr2=0x55d0ac4a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 -- 172.21.15.111:0/1397200858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 msgr2=0x55d0ac49bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 0x55d0ac49bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 -- 172.21.15.111:0/1397200858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ac49c2e0 con 0x55d0ac49ffb0 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64efe00640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 0x55d0ac49bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f64e80bbf50 tx=0x7f64e80f3810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cc445d540a00ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64e6a00640 1 -- 172.21.15.111:0/1397200858 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8049040 con 0x55d0ac49ffb0 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64e6a00640 1 -- 172.21.15.111:0/1397200858 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8031e40 con 0x55d0ac49ffb0 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.366+0000 7f64e6a00640 1 -- 172.21.15.111:0/1397200858 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e80c0600 con 0x55d0ac49ffb0 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- 172.21.15.111:0/1397200858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 msgr2=0x55d0ac4b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f64e80bbf50 tx=0x7f64e80f3810 comp rx=0 tx=0).stop 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- 172.21.15.111:0/1397200858 shutdown_connections 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac4ab190 0x55d0ac4a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4a08d0 0x55d0ac49bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1397200858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b6c80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- 172.21.15.111:0/1397200858 >> 172.21.15.111:0/1397200858 conn(0x55d0ac49c4c0 msgr2=0x55d0ac4a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- 172.21.15.111:0/1397200858 shutdown_connections 2024-04-24T22:47:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- 172.21.15.111:0/1397200858 wait complete. 2024-04-24T22:47:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.367+0000 7f64ffd94740 1 -- start start 2024-04-24T22:47:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 Processor -- start 2024-04-24T22:47:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b1ec0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b3330 con 0x55d0ac4ab190 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81b2590 con 0x55d0ac49ffb0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64ffd94740 10 client.0 _ll_register_callbacks cb 0x7f64e0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33284/0 (socket says 172.21.15.111:33284) 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64efe00640 1 -- 172.21.15.111:0/1936584356 learned_addr learned my addr 172.21.15.111:0/1936584356 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.368+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64f6800640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 -- 172.21.15.111:0/1936584356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 msgr2=0x55d0ac4b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 -- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 msgr2=0x55d0ac4b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 -- 172.21.15.111:0/1936584356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ac4b0990 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64f6800640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64efe00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64e001c510 tx=0x7f64e0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2294db942b66658 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e001c590 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d0ac40b340 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.369+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f64e001c730 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f64e0036050 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 10 client.15615 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d0ac5221b0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d0ac40b340 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f64e00035f0 con 0x55d0ac4a08d0 2024-04-24T22:47:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 20 client.15615 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac522900 0x55d0ac524d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 4 client.15615 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.370+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0ac3897a0 con 0x55d0ac522900 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.371+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac522900 0x55d0ac524d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.371+0000 7f64f5e00640 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac522900 0x55d0ac524d20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e2ba59e6e63efd server_cookie=eddc2396e7dc54c5 in_seq=0 out_seq=0 2024-04-24T22:47:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.371+0000 7f64dbe00640 10 client.15615 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64dbe00640 1 -- 172.21.15.111:0/1936584356 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0ac3897a0 con 0x55d0ac522900 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64dbe00640 10 client.15615 handle_command_reply: tid=0 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 client.15615 shutdown 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 msgr2=0x55d0ac4b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64e001c510 tx=0x7f64e0005570 comp rx=0 tx=0).stop 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 shutdown_connections 2024-04-24T22:47:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d0ac522900 0x55d0ac524d20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0ac4ab190 0x55d0ac4b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0ac4a08d0 0x55d0ac4b36d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 --2- 172.21.15.111:0/1936584356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d0ac49ffb0 0x55d0ac4b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64dbe00640 0 client.15615 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 >> 172.21.15.111:0/1936584356 conn(0x55d0ac49c4c0 msgr2=0x55d0ac4a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 shutdown_connections 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 1 -- 172.21.15.111:0/1936584356 wait complete. 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.386+0000 7f64ffd94740 20 client.15615 trim_cache size 0 max 16384 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 msgr2=0x7f64dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 0x7f64dc0a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64f0020000 tx=0x7f64f0021000 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 msgr2=0x7f64f819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f64e802a940 tx=0x7f64e8003b20 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 shutdown_connections 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f810dc10 0x7f64f81a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f8108600 0x7f64f819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 --2- 172.21.15.111:0/1725877075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64f8107c80 0x7f64f819e490 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 >> 172.21.15.111:0/1725877075 conn(0x7f64f8077520 msgr2=0x7f64f8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 shutdown_connections 2024-04-24T22:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:03.387+0000 7f64f7200640 1 -- 172.21.15.111:0/1725877075 wait complete. 2024-04-24T22:47:04.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:04 smithi186 ceph-mon[32753]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-04-24T22:47:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:04 smithi111 ceph-mon[27746]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-04-24T22:47:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:04 smithi171 ceph-mon[32337]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-04-24T22:47:06.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:06 smithi186 ceph-mon[32753]: pgmap v157: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:06 smithi111 ceph-mon[27746]: pgmap v157: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:06 smithi171 ceph-mon[32337]: pgmap v157: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:08.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:08 smithi186 ceph-mon[32753]: pgmap v158: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:08 smithi111 ceph-mon[27746]: pgmap v158: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:08.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:08 smithi171 ceph-mon[32337]: pgmap v158: 97 pgs: 97 active+clean; 3.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 54 KiB/s wr, 24 op/s 2024-04-24T22:47:10.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:10 smithi186 ceph-mon[32753]: pgmap v159: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-04-24T22:47:10.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:10 smithi111 ceph-mon[27746]: pgmap v159: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-04-24T22:47:10.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:10 smithi171 ceph-mon[32337]: pgmap v159: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-04-24T22:47:12.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:12 smithi186 ceph-mon[32753]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-24T22:47:12.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:12 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:12 smithi111 ceph-mon[27746]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-24T22:47:12.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:12 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:12.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:12 smithi171 ceph-mon[32337]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-24T22:47:12.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:12 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:14 smithi186 ceph-mon[32753]: pgmap v161: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-04-24T22:47:14.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:14 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:14.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:14 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:14.500 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:14 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:47:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:14 smithi111 ceph-mon[27746]: pgmap v161: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-04-24T22:47:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:14 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:14 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:14 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:47:14.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:14 smithi171 ceph-mon[32337]: pgmap v161: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-04-24T22:47:14.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:14 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:14.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:14 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:14.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:14 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:47:16.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:16 smithi186 ceph-mon[32753]: pgmap v162: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 -- 172.21.15.111:0/3346092519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 msgr2=0x7fbca8109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 --2- 172.21.15.111:0/3346092519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca8109d20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009930 tx=0x7fbc9802a440 comp rx=0 tx=0).stop 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 -- 172.21.15.111:0/3346092519 shutdown_connections 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 --2- 172.21.15.111:0/3346092519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca81042f0 0x7fbca8110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 --2- 172.21.15.111:0/3346092519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca8103930 0x7fbca8103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 --2- 172.21.15.111:0/3346092519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca8109d20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 -- 172.21.15.111:0/3346092519 >> 172.21.15.111:0/3346092519 conn(0x7fbca80ff440 msgr2=0x7fbca8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 -- 172.21.15.111:0/3346092519 shutdown_connections 2024-04-24T22:47:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.531+0000 7fbcacc00640 1 -- 172.21.15.111:0/3346092519 wait complete. 2024-04-24T22:47:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:16 smithi111 ceph-mon[27746]: pgmap v162: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.532+0000 7fbcacc00640 1 Processor -- start 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.532+0000 7fbcacc00640 1 -- start start 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 0x7fbca819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8113d70 con 0x7fbca8109940 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8113bf0 con 0x7fbca8103930 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbcacc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8113ef0 con 0x7fbca81042f0 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbca5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbca6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbca5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36090/0 (socket says 172.21.15.111:36090) 2024-04-24T22:47:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbca5e00640 1 -- 172.21.15.111:0/1292198465 learned_addr learned my addr 172.21.15.111:0/1292198465 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.533+0000 7fbca5400640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 0x7fbca819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 -- 172.21.15.111:0/1292198465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 msgr2=0x7fbca819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 0x7fbca819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 -- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 msgr2=0x7fbca819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 -- 172.21.15.111:0/1292198465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca819f1c0 con 0x7fbca8103930 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca6800640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5400640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 0x7fbca819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbca5e00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc9802a960 tx=0x7fbc98003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbc9ea00640 1 -- 172.21.15.111:0/1292198465 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9803f070 con 0x7fbca8103930 2024-04-24T22:47:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca819f450 con 0x7fbca8103930 2024-04-24T22:47:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbc9ea00640 1 -- 172.21.15.111:0/1292198465 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98031c90 con 0x7fbca8103930 2024-04-24T22:47:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbc9ea00640 1 -- 172.21.15.111:0/1292198465 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98035030 con 0x7fbca8103930 2024-04-24T22:47:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.534+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca81a6d40 con 0x7fbca8103930 2024-04-24T22:47:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 Processor -- start 2024-04-24T22:47:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 -- start start 2024-04-24T22:47:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbc9ea00640 1 -- 172.21.15.111:0/1292198465 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbc98042070 con 0x7fbca8103930 2024-04-24T22:47:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 0x557e2408faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e23fe8ab0 con 0x557e24085720 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557e23f91640 con 0x557e24084e00 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557e23fe4a70 con 0x557e2408ffe0 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbca5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 0x557e2408faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36094/0 (socket says 172.21.15.111:36094) 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.538+0000 7fbca6800640 1 -- 172.21.15.111:0/2621275042 learned_addr learned my addr 172.21.15.111:0/2621275042 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca5400640 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 -- 172.21.15.111:0/2621275042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 msgr2=0x557e240923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 -- 172.21.15.111:0/2621275042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 msgr2=0x557e2408faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 0x557e2408faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 -- 172.21.15.111:0/2621275042 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e23cfd210 con 0x557e24084e00 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca5e00640 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 0x557e2408faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbca6800640 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbca0011790 tx=0x7fbca0011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1043fba3fd6bfbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.539+0000 7fbc93000640 1 -- 172.21.15.111:0/2621275042 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004d20 con 0x557e24084e00 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbc93000640 1 -- 172.21.15.111:0/2621275042 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca0003970 con 0x557e24084e00 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbc93000640 1 -- 172.21.15.111:0/2621275042 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004560 con 0x557e24084e00 2024-04-24T22:47:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbc9ea00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 0x7fbc8c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbc9ea00640 1 -- 172.21.15.111:0/1292198465 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbc980efdd0 con 0x7fbca8103930 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbca5e00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 0x7fbc8c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2621275042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 msgr2=0x557e240851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbca0011790 tx=0x7fbca0011c50 comp rx=0 tx=0).stop 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2621275042 shutdown_connections 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24085720 0x557e2408faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2621275042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e240851e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.540+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2621275042 >> 172.21.15.111:0/2621275042 conn(0x557e24081310 msgr2=0x557e240816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.541+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2621275042 shutdown_connections 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.541+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2621275042 wait complete. 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.541+0000 7fbca5e00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 0x7fbc8c0a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x557e240863a0 tx=0x7fbc980f3880 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.541+0000 7fbcafb6f740 1 -- start start 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.541+0000 7fbcafb6f740 1 Processor -- start 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e2408cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e24092c10 con 0x557e2408ffe0 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557e23fa4990 con 0x557e24084e00 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557e24092d90 con 0x557e24085720 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbcafb6f740 10 client.0 _ll_register_callbacks cb 0x557e23fa6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e2408cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e2408cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36108/0 (socket says 172.21.15.111:36108) 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca6800640 1 -- 172.21.15.111:0/2202257665 learned_addr learned my addr 172.21.15.111:0/2202257665 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca6800640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 -- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 msgr2=0x557e2408cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e2408cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 -- 172.21.15.111:0/2202257665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 msgr2=0x557e2408da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.542+0000 7fbca5400640 1 -- 172.21.15.111:0/2202257665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e23dc9620 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbca5400640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408d4e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbca81019e0 tx=0x7fbc9400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498f655ca452f44a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbc87400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94012110 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbc94004050 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca0003970 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbc940122b0 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbcafb6f740 10 client.25708 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e2409ebe0 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.543+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca0003970 con 0x557e24085720 2024-04-24T22:47:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbc940254e0 con 0x557e24085720 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 10 client.25708 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 20 client.25708 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e240a00c0 0x557e240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 4 client.25708 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e23ff5ff0 con 0x557e240a00c0 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.544+0000 7fbca6800640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e240a00c0 0x557e240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.545+0000 7fbca6800640 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e240a00c0 0x557e240a24e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83023aa402e0d71 server_cookie=c8eccb26c06acb37 in_seq=0 out_seq=0 2024-04-24T22:47:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.545+0000 7fbc87400640 10 client.25708 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbc87400640 1 -- 172.21.15.111:0/2202257665 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e23ff5ff0 con 0x557e240a00c0 2024-04-24T22:47:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbc87400640 10 client.25708 handle_command_reply: tid=0 2024-04-24T22:47:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 client.25708 shutdown 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 msgr2=0x557e2408d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408d4e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbca81019e0 tx=0x7fbc9400f790 comp rx=0 tx=0).stop 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 shutdown_connections 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e240a00c0 0x557e240a24e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408d4e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2202257665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e2408cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbc87400640 0 client.25708 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 >> 172.21.15.111:0/2202257665 conn(0x557e24081310 msgr2=0x557e24087090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 shutdown_connections 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2202257665 wait complete. 2024-04-24T22:47:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.546+0000 7fbcafb6f740 20 client.25708 trim_cache size 0 max 16384 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.562+0000 7fbcafb6f740 1 Processor -- start 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.562+0000 7fbcafb6f740 1 -- start start 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24085720 0x557e24080c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81b1e80 con 0x557e24084e00 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81af750 con 0x557e24085720 2024-04-24T22:47:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81ad5a0 con 0x557e2408ffe0 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53608/0 (socket says 172.21.15.111:53608) 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 -- 172.21.15.111:0/2544181265 learned_addr learned my addr 172.21.15.111:0/2544181265 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca5400640 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24085720 0x557e24080c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 -- 172.21.15.111:0/2544181265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 msgr2=0x557e240a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 -- 172.21.15.111:0/2544181265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24085720 msgr2=0x557e24080c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24085720 0x557e24080c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.563+0000 7fbca6800640 1 -- 172.21.15.111:0/2544181265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e24081140 con 0x557e24084e00 2024-04-24T22:47:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.564+0000 7fbca6800640 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbca0011730 tx=0x7fbca0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef34b0871b96b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.564+0000 7fbc92600640 1 -- 172.21.15.111:0/2544181265 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0033070 con 0x557e24084e00 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.564+0000 7fbc92600640 1 -- 172.21.15.111:0/2544181265 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca002e050 con 0x557e24084e00 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2544181265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 msgr2=0x557e2409bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbca0011730 tx=0x7fbca0009500 comp rx=0 tx=0).stop 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2544181265 shutdown_connections 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e2408ffe0 0x557e240a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24085720 0x557e24080c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2544181265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e24084e00 0x557e2409bb00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2544181265 >> 172.21.15.111:0/2544181265 conn(0x557e24081310 msgr2=0x557e24087040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2544181265 shutdown_connections 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2544181265 wait complete. 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.565+0000 7fbcafb6f740 1 -- start start 2024-04-24T22:47:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 Processor -- start 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e24089e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81adc10 con 0x557e2408ffe0 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81af080 con 0x557e24084e00 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81ae2e0 con 0x557e24085720 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbcafb6f740 10 client.0 _ll_register_callbacks cb 0x7fbc94009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbca6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbca6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53620/0 (socket says 172.21.15.111:53620) 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.566+0000 7fbca6800640 1 -- 172.21.15.111:0/2229816997 learned_addr learned my addr 172.21.15.111:0/2229816997 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e24089e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca5400640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 -- 172.21.15.111:0/2229816997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 msgr2=0x557e2408a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 -- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 msgr2=0x557e24089e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e24089e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 -- 172.21.15.111:0/2229816997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2410df50 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbca6800640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbca000ff30 tx=0x7fbca0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e50a64f086b223b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbc87400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0033040 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fbca0004b20 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.567+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca002e050 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbca001d040 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 10 client.15633 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e2410f260 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca002e050 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbca0039070 con 0x557e2408ffe0 2024-04-24T22:47:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 20 client.15633 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e2410f6e0 0x557e24111ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 4 client.15633 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.568+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e23ff0340 con 0x557e2410f6e0 2024-04-24T22:47:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.569+0000 7fbca5400640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e2410f6e0 0x557e24111ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.570+0000 7fbca5400640 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e2410f6e0 0x557e24111ad0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcf7ab509c728db server_cookie=e9da9b33f6a03b0b in_seq=0 out_seq=0 2024-04-24T22:47:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.570+0000 7fbc87400640 10 client.15633 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.583+0000 7fbc87400640 1 -- 172.21.15.111:0/2229816997 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e23ff0340 con 0x557e2410f6e0 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.583+0000 7fbc87400640 10 client.15633 handle_command_reply: tid=0 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 client.15633 shutdown 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 msgr2=0x557e2408a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbca000ff30 tx=0x7fbca0009fa0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 shutdown_connections 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x557e2410f6e0 0x557e24111ad0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e2408ffe0 0x557e2408a880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e24085720 0x557e2408a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 --2- 172.21.15.111:0/2229816997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557e24084e00 0x557e24089e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbc87400640 0 client.15633 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 >> 172.21.15.111:0/2229816997 conn(0x557e24081310 msgr2=0x557e2408cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 shutdown_connections 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 1 -- 172.21.15.111:0/2229816997 wait complete. 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.584+0000 7fbcafb6f740 20 client.15633 trim_cache size 0 max 16384 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 msgr2=0x7fbc8c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 0x7fbc8c0a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x557e240863a0 tx=0x7fbc980f3880 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 msgr2=0x7fbca819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc9802a960 tx=0x7fbc98003b20 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 shutdown_connections 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbca8109940 0x7fbca819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc8c0a2da0 0x7fbc8c0a5260 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca81042f0 0x7fbca819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 --2- 172.21.15.111:0/1292198465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbca8103930 0x7fbca819a1d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 >> 172.21.15.111:0/1292198465 conn(0x7fbca80ff440 msgr2=0x7fbca80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.585+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 shutdown_connections 2024-04-24T22:47:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:16.586+0000 7fbcacc00640 1 -- 172.21.15.111:0/1292198465 wait complete. 2024-04-24T22:47:16.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:16 smithi171 ceph-mon[32337]: pgmap v162: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:18.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:18 smithi186 ceph-mon[32753]: pgmap v163: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:18.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:18 smithi111 ceph-mon[27746]: pgmap v163: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:18.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:18 smithi171 ceph-mon[32337]: pgmap v163: 97 pgs: 97 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:47:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:20 smithi186 ceph-mon[32753]: pgmap v164: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-24T22:47:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:20 smithi111 ceph-mon[27746]: pgmap v164: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-24T22:47:20.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:20 smithi171 ceph-mon[32337]: pgmap v164: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-24T22:47:22.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:22 smithi111 ceph-mon[27746]: pgmap v165: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-24T22:47:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:22 smithi171 ceph-mon[32337]: pgmap v165: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-24T22:47:22.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:22 smithi186 ceph-mon[32753]: pgmap v165: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-24T22:47:24.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:24 smithi111 ceph-mon[27746]: pgmap v166: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:24 smithi171 ceph-mon[32337]: pgmap v166: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:24.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:24 smithi186 ceph-mon[32753]: pgmap v166: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:26.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:26 smithi111 ceph-mon[27746]: pgmap v167: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:26 smithi171 ceph-mon[32337]: pgmap v167: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:26 smithi186 ceph-mon[32753]: pgmap v167: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:28.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:28 smithi111 ceph-mon[27746]: pgmap v168: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:28 smithi171 ceph-mon[32337]: pgmap v168: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:28.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:28 smithi186 ceph-mon[32753]: pgmap v168: 97 pgs: 97 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-24T22:47:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 -- 172.21.15.111:0/335315161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4109930 msgr2=0x7fa2b4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 --2- 172.21.15.111:0/335315161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4109930 0x7fa2b4109d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009890 tx=0x7fa2a402a370 comp rx=0 tx=0).stop 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 -- 172.21.15.111:0/335315161 shutdown_connections 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 --2- 172.21.15.111:0/335315161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b41107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 --2- 172.21.15.111:0/335315161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4103920 0x7fa2b4103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 --2- 172.21.15.111:0/335315161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4109930 0x7fa2b4109d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 -- 172.21.15.111:0/335315161 >> 172.21.15.111:0/335315161 conn(0x7fa2b40ff430 msgr2=0x7fa2b4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 -- 172.21.15.111:0/335315161 shutdown_connections 2024-04-24T22:47:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.730+0000 7fa2b9600640 1 -- 172.21.15.111:0/335315161 wait complete. 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.731+0000 7fa2b9600640 1 Processor -- start 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 -- start start 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 0x7fa2b419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4109930 0x7fa2b419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4113d40 con 0x7fa2b4109930 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4113bc0 con 0x7fa2b41042e0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b9600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4113ec0 con 0x7fa2b4103920 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 0x7fa2b419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:50252/0 (socket says 172.21.15.111:50252) 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 0x7fa2b419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:35202/0 (socket says 172.21.15.111:35202) 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b2000640 1 -- 172.21.15.111:0/1092380102 learned_addr learned my addr 172.21.15.111:0/1092380102 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.732+0000 7fa2b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4109930 0x7fa2b419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 -- 172.21.15.111:0/1092380102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 msgr2=0x7fa2b419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 0x7fa2b419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 -- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4109930 msgr2=0x7fa2b419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4109930 0x7fa2b419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 -- 172.21.15.111:0/1092380102 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b419f150 con 0x7fa2b41042e0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b2000640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e9c0 tx=0x7fa2a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2ab400640 1 -- 172.21.15.111:0/1092380102 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000cde0 con 0x7fa2b41042e0 2024-04-24T22:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b419f440 con 0x7fa2b41042e0 2024-04-24T22:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2ab400640 1 -- 172.21.15.111:0/1092380102 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00035f0 con 0x7fa2b41042e0 2024-04-24T22:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2ab400640 1 -- 172.21.15.111:0/1092380102 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0010650 con 0x7fa2b41042e0 2024-04-24T22:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.733+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b41a6cd0 con 0x7fa2b41042e0 2024-04-24T22:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.736+0000 7fa2bc5bb740 1 Processor -- start 2024-04-24T22:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 -- start start 2024-04-24T22:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d297aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d29a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56364d1f0ab0 con 0x56364d28ce00 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56364d199640 con 0x56364d297fe0 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56364d1eca70 con 0x56364d28d720 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2ab400640 1 -- 172.21.15.111:0/1092380102 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa2a00037a0 con 0x7fa2b41042e0 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47998/0 (socket says 172.21.15.111:47998) 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d297aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.737+0000 7fa2b3400640 1 -- 172.21.15.111:0/340726945 learned_addr learned my addr 172.21.15.111:0/340726945 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2000640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d29a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 -- 172.21.15.111:0/340726945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 msgr2=0x56364d29a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d29a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 -- 172.21.15.111:0/340726945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 msgr2=0x56364d28d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 -- 172.21.15.111:0/340726945 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364cf05210 con 0x56364d28d720 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2000640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d29a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b3400640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d297aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40057c0 tx=0x7fa2a4005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc09c5df7b17f2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa29f400640 1 -- 172.21.15.111:0/340726945 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4041020 con 0x56364d28d720 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa29f400640 1 -- 172.21.15.111:0/340726945 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4030e40 con 0x56364d28d720 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.738+0000 7fa29f400640 1 -- 172.21.15.111:0/340726945 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4037040 con 0x56364d28d720 2024-04-24T22:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2ab400640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 0x7fa2980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2ab400640 1 -- 172.21.15.111:0/1092380102 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a00cf7a0 con 0x7fa2b41042e0 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/340726945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 msgr2=0x56364d297aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d297aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40057c0 tx=0x7fa2a4005b20 comp rx=0 tx=0).stop 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2b3400640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 0x7fa2980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/340726945 shutdown_connections 2024-04-24T22:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d29a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d297aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/340726945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d28d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/340726945 >> 172.21.15.111:0/340726945 conn(0x56364d289310 msgr2=0x56364d2896f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/340726945 shutdown_connections 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.739+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/340726945 wait complete. 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.740+0000 7fa2b3400640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 0x7fa2980a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009d60 tx=0x7fa2ac009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.740+0000 7fa2bc5bb740 1 -- start start 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.740+0000 7fa2bc5bb740 1 Processor -- start 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d2959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56364d29abb0 con 0x56364d28ce00 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56364d1ac990 con 0x56364d297fe0 2024-04-24T22:47:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56364d29ad30 con 0x56364d28d720 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2bc5bb740 10 client.0 _ll_register_callbacks cb 0x56364d1ae950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48014/0 (socket says 172.21.15.111:48014) 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:35206/0 (socket says 172.21.15.111:35206) 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2a00640 1 -- 172.21.15.111:0/647786390 learned_addr learned my addr 172.21.15.111:0/647786390 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b3400640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d2959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 -- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 msgr2=0x56364d2959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d2959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 -- 172.21.15.111:0/647786390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 msgr2=0x56364d294f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2000640 1 -- 172.21.15.111:0/647786390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364cfd1620 con 0x56364d28d720 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.741+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa2b2000640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00ccec0 tx=0x7fa2a009b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c114991b5a5bb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa293e00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00a4900 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a00cd140 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a4030e40 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2a00cd340 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa2bc5bb740 10 client.25723 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.742+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56364d2a69f0 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4030e40 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2a007aab0 con 0x56364d28d720 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 10 client.25723 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 20 client.25723 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d2a7e60 0x56364d2aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 4 client.25723 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364d1fdff0 con 0x56364d2a7e60 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.743+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d2a7e60 0x56364d2aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.744+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d2a7e60 0x56364d2aa290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482f40008b6a1249 server_cookie=d1663c7758820853 in_seq=0 out_seq=0 2024-04-24T22:47:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.744+0000 7fa293e00640 10 client.25723 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa293e00640 1 -- 172.21.15.111:0/647786390 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56364d1fdff0 con 0x56364d2a7e60 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa293e00640 10 client.25723 handle_command_reply: tid=0 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 client.25723 shutdown 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 msgr2=0x56364d295470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00ccec0 tx=0x7fa2a009b6e0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 shutdown_connections 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d2a7e60 0x56364d2aa290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d2959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d295470 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/647786390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d294f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa293e00640 0 client.25723 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 >> 172.21.15.111:0/647786390 conn(0x56364d289310 msgr2=0x56364d28f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 shutdown_connections 2024-04-24T22:47:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/647786390 wait complete. 2024-04-24T22:47:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.745+0000 7fa2bc5bb740 20 client.25723 trim_cache size 0 max 16384 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.761+0000 7fa2bc5bb740 1 Processor -- start 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.761+0000 7fa2bc5bb740 1 -- start start 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28ce00 0x56364d2a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 0x56364d2afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41b1db0 con 0x56364d297fe0 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41af680 con 0x56364d28d720 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41ad4d0 con 0x56364d28ce00 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48020/0 (socket says 172.21.15.111:48020) 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b3400640 1 -- 172.21.15.111:0/3045769211 learned_addr learned my addr 172.21.15.111:0/3045769211 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 0x56364d2afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28ce00 0x56364d2a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 -- 172.21.15.111:0/3045769211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28ce00 msgr2=0x56364d2a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28ce00 0x56364d2a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 -- 172.21.15.111:0/3045769211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 msgr2=0x56364d2b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.762+0000 7fa2b2000640 1 -- 172.21.15.111:0/3045769211 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364d287890 con 0x56364d28d720 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.763+0000 7fa2b3400640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.763+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 0x56364d2afba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0099f40 tx=0x7fa2a00a01c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c1a866d12970be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.763+0000 7fa29ea00640 1 -- 172.21.15.111:0/3045769211 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a009bc90 con 0x56364d28d720 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.763+0000 7fa29ea00640 1 -- 172.21.15.111:0/3045769211 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a007aab0 con 0x56364d28d720 2024-04-24T22:47:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.763+0000 7fa29ea00640 1 -- 172.21.15.111:0/3045769211 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a009b0e0 con 0x56364d28d720 2024-04-24T22:47:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3045769211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 msgr2=0x56364d2afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 0x56364d2afba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0099f40 tx=0x7fa2a00a01c0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3045769211 shutdown_connections 2024-04-24T22:47:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d297fe0 0x56364d2b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d28d720 0x56364d2afba0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3045769211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28ce00 0x56364d2a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3045769211 >> 172.21.15.111:0/3045769211 conn(0x56364d289310 msgr2=0x56364d284000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3045769211 shutdown_connections 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3045769211 wait complete. 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 -- start start 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.764+0000 7fa2bc5bb740 1 Processor -- start 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d292570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d30d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41adb40 con 0x56364d28ce00 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41aefb0 con 0x56364d297fe0 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41ae210 con 0x56364d28d720 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2bc5bb740 10 client.0 _ll_register_callbacks cb 0x7fa2a0004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d292570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2b2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48024/0 (socket says 172.21.15.111:48024) 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.765+0000 7fa2b2a00640 1 -- 172.21.15.111:0/3285573498 learned_addr learned my addr 172.21.15.111:0/3285573498 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b3400640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d30d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 -- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 msgr2=0x56364d30d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d30d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 -- 172.21.15.111:0/3285573498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 msgr2=0x56364d292030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 -- 172.21.15.111:0/3285573498 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364d30d950 con 0x56364d28d720 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b3400640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d30d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2b2000640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d292570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00cd4c0 tx=0x7fa2a00cfa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94b54d2fc3138d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa293e00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00cfb40 con 0x56364d28d720 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a009b2f0 con 0x56364d28d720 2024-04-24T22:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.766+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a007aab0 con 0x56364d28d720 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2a009bc90 con 0x56364d28d720 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 10 client.25729 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56364d30eb00 con 0x56364d28d720 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a007aab0 con 0x56364d28d720 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2a009ad50 con 0x56364d28d720 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 10 client.25729 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 20 client.25729 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d30f1a0 0x56364d3115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 4 client.25729 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.767+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56364d1f8340 con 0x56364d30f1a0 2024-04-24T22:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.768+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d30f1a0 0x56364d3115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.768+0000 7fa2b2a00640 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d30f1a0 0x56364d3115c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25f02216d912a76 server_cookie=87187a8a93caa1a7 in_seq=0 out_seq=0 2024-04-24T22:47:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.768+0000 7fa293e00640 10 client.25729 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa293e00640 1 -- 172.21.15.111:0/3285573498 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56364d1f8340 con 0x56364d30f1a0 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa293e00640 10 client.25729 handle_command_reply: tid=0 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 client.25729 shutdown 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 msgr2=0x56364d292570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d292570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00cd4c0 tx=0x7fa2a00cfa90 comp rx=0 tx=0).stop 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 shutdown_connections 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56364d30f1a0 0x56364d3115c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56364d297fe0 0x56364d30d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56364d28d720 0x56364d292570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 --2- 172.21.15.111:0/3285573498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56364d28ce00 0x56364d292030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa293e00640 0 client.25729 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.784+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 >> 172.21.15.111:0/3285573498 conn(0x56364d289310 msgr2=0x56364d294ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 shutdown_connections 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2bc5bb740 1 -- 172.21.15.111:0/3285573498 wait complete. 2024-04-24T22:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2bc5bb740 20 client.25729 trim_cache size 0 max 16384 2024-04-24T22:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 msgr2=0x7fa2980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 0x7fa2980a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009d60 tx=0x7fa2ac009500 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 msgr2=0x7fa2b419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.785+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e9c0 tx=0x7fa2a000ee80 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 shutdown_connections 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2b4109930 0x7fa2b419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41042e0 0x7fa2b419a6a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fa2980a2da0 0x7fa2980a5260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 --2- 172.21.15.111:0/1092380102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b4103920 0x7fa2b419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 >> 172.21.15.111:0/1092380102 conn(0x7fa2b40ff430 msgr2=0x7fa2b40ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 shutdown_connections 2024-04-24T22:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:29.786+0000 7fa2b9600640 1 -- 172.21.15.111:0/1092380102 wait complete. 2024-04-24T22:47:30.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:30 smithi111 ceph-mon[27746]: pgmap v169: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-04-24T22:47:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:30 smithi171 ceph-mon[32337]: pgmap v169: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-04-24T22:47:30.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:30 smithi186 ceph-mon[32753]: pgmap v169: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-04-24T22:47:31.665 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.665 INFO:tasks.workunit.client.0.smithi111.stdout:189/600: open fd = 3 2024-04-24T22:47:31.665 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.665 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.665 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:190/600: open fd = 3 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:191/600: open fd = 3 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:192/600: open fd = 3 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:193/600: open fd = 3 2024-04-24T22:47:31.666 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:194/600: open fd = 3 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:195/600: open fd = 3 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:196/600: open fd = 3 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.667 INFO:tasks.workunit.client.0.smithi111.stdout:197/600: open fd = 3 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:198/600: open fd = 3 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:199/600: open fd = 3 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:200/600: open fd = 3 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.668 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:201/600: open fd = 3 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:202/600: open fd = 3 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:203/600: open fd = 3 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:204/600: open fd = 3 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.669 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:205/600: open fd = 3 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:206/600: open fd = 3 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:207/600: open fd = 3 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.670 INFO:tasks.workunit.client.0.smithi111.stdout:208/600: open fd = 3 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:209/600: open fd = 3 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:210/600: open fd = 3 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:211/600: open fd = 3 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.671 INFO:tasks.workunit.client.0.smithi111.stdout:212/600: open fd = 3 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:213/600: open fd = 3 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:214/600: open fd = 3 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:215/600: open fd = 3 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.672 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:216/600: open fd = 3 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:217/600: open fd = 3 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:218/600: open fd = 3 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:219/600: open fd = 3 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.673 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:220/600: open fd = 3 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:221/600: open fd = 3 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:222/600: open fd = 3 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.674 INFO:tasks.workunit.client.0.smithi111.stdout:223/600: open fd = 3 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:224/600: open fd = 3 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:225/600: open fd = 3 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:226/600: open fd = 3 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.675 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:227/600: open fd = 3 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:228/600: open fd = 3 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:229/600: open fd = 3 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:230/600: open fd = 3 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.676 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:231/600: open fd = 3 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:232/600: open fd = 3 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:233/600: open fd = 3 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:234/600: open fd = 3 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.677 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:235/600: open fd = 3 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:236/600: open fd = 3 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:237/600: open fd = 3 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.678 INFO:tasks.workunit.client.0.smithi111.stdout:238/600: open fd = 3 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:239/600: open fd = 3 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:240/600: open fd = 3 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:241/600: open fd = 3 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.679 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:242/600: open fd = 3 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:243/600: open fd = 3 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:244/600: open fd = 3 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:245/600: open fd = 3 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.680 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:246/600: open fd = 3 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:247/600: open fd = 3 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:248/600: open fd = 3 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:249/600: open fd = 3 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.681 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.682 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:47:31.682 INFO:tasks.workunit.client.0.smithi111.stdout:250/600: open fd = 3 2024-04-24T22:47:31.682 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:31.682 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:47:32.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:32 smithi111 ceph-mon[27746]: pgmap v170: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:47:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:32 smithi171 ceph-mon[32337]: pgmap v170: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:47:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:32 smithi186 ceph-mon[32753]: pgmap v170: 97 pgs: 97 active+clean; 4.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:47:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:34 smithi171 ceph-mon[32337]: pgmap v171: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-04-24T22:47:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:34 smithi186 ceph-mon[32753]: pgmap v171: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-04-24T22:47:34.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:34 smithi111 ceph-mon[27746]: pgmap v171: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-04-24T22:47:36.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:36 smithi186 ceph-mon[32753]: pgmap v172: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:36.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:36 smithi171 ceph-mon[32337]: pgmap v172: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:36.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:36 smithi111 ceph-mon[27746]: pgmap v172: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:38.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:38 smithi171 ceph-mon[32337]: pgmap v173: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:38 smithi186 ceph-mon[32753]: pgmap v173: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:38.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:38 smithi111 ceph-mon[27746]: pgmap v173: 97 pgs: 97 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:47:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:40 smithi171 ceph-mon[32337]: pgmap v174: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:47:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:40 smithi186 ceph-mon[32753]: pgmap v174: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:47:40.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:40 smithi111 ceph-mon[27746]: pgmap v174: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:47:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:42 smithi171 ceph-mon[32337]: pgmap v175: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:47:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:42 smithi186 ceph-mon[32753]: pgmap v175: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:47:42.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:42 smithi111 ceph-mon[27746]: pgmap v175: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:47:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 -- 172.21.15.111:0/4134440977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c109760 msgr2=0x7f7c8c109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 --2- 172.21.15.111:0/4134440977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c109760 0x7f7c8c109b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009900 tx=0x7f7c7c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 -- 172.21.15.111:0/4134440977 shutdown_connections 2024-04-24T22:47:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 --2- 172.21.15.111:0/4134440977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c104110 0x7f7c8c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 --2- 172.21.15.111:0/4134440977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c103750 0x7f7c8c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 --2- 172.21.15.111:0/4134440977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c109760 0x7f7c8c109b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 -- 172.21.15.111:0/4134440977 >> 172.21.15.111:0/4134440977 conn(0x7f7c8c0ff260 msgr2=0x7f7c8c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 -- 172.21.15.111:0/4134440977 shutdown_connections 2024-04-24T22:47:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.929+0000 7f7c90a00640 1 -- 172.21.15.111:0/4134440977 wait complete. 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 Processor -- start 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 -- start start 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c103750 0x7f7c8c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 0x7f7c8c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c113bc0 con 0x7f7c8c109760 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c113a40 con 0x7f7c8c103750 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.930+0000 7f7c90a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c113d40 con 0x7f7c8c104110 2024-04-24T22:47:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 0x7f7c8c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59808/0 (socket says 172.21.15.111:59808) 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8be00640 1 -- 172.21.15.111:0/977468200 learned_addr learned my addr 172.21.15.111:0/977468200 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8b400640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c103750 0x7f7c8c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 -- 172.21.15.111:0/977468200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c103750 msgr2=0x7f7c8c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c103750 0x7f7c8c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 -- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 msgr2=0x7f7c8c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 -- 172.21.15.111:0/977468200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c19f050 con 0x7f7c8c104110 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8be00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.931+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 0x7f7c8c19a5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.932+0000 7f7c83e00640 1 -- 172.21.15.111:0/977468200 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800ccc0 con 0x7f7c8c104110 2024-04-24T22:47:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.932+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c19f340 con 0x7f7c8c104110 2024-04-24T22:47:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.932+0000 7f7c83e00640 1 -- 172.21.15.111:0/977468200 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c78005020 con 0x7f7c8c104110 2024-04-24T22:47:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.932+0000 7f7c83e00640 1 -- 172.21.15.111:0/977468200 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780049e0 con 0x7f7c8c104110 2024-04-24T22:47:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.932+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1a6bd0 con 0x7f7c8c104110 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 Processor -- start 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 -- start start 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d91390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d9c190 0x55d456d9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d456cf4ab0 con 0x55d456d918d0 2024-04-24T22:47:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d456c9d640 con 0x55d456d9c190 2024-04-24T22:47:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.935+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d456cf0a70 con 0x55d456d90fb0 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59820/0 (socket says 172.21.15.111:59820) 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d91390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 -- 172.21.15.111:0/2403984221 learned_addr learned my addr 172.21.15.111:0/2403984221 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c83e00640 1 -- 172.21.15.111:0/977468200 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7c78005560 con 0x7f7c8c104110 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 -- 172.21.15.111:0/2403984221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 msgr2=0x55d456d91390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d91390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 -- 172.21.15.111:0/2403984221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d9c190 msgr2=0x55d456d9e580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d9c190 0x55d456d9e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 -- 172.21.15.111:0/2403984221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d456a09210 con 0x55d456d918d0 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8be00640 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d91390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.936+0000 7f7c8b400640 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009230 tx=0x7f7c7c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8606b5782d08e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c77e00640 1 -- 172.21.15.111:0/2403984221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c035040 con 0x55d456d918d0 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c77e00640 1 -- 172.21.15.111:0/2403984221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c005520 con 0x55d456d918d0 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c77e00640 1 -- 172.21.15.111:0/2403984221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0035f0 con 0x55d456d918d0 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c83e00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 0x7f7c700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c94e4a740 1 -- 172.21.15.111:0/2403984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 msgr2=0x55d456d9bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.937+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009230 tx=0x7f7c7c005e00 comp rx=0 tx=0).stop 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c83e00640 1 -- 172.21.15.111:0/977468200 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f7c780cf370 con 0x7f7c8c104110 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 -- 172.21.15.111:0/2403984221 shutdown_connections 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d9c190 0x55d456d9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d918d0 0x55d456d9bc50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/2403984221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d91390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 -- 172.21.15.111:0/2403984221 >> 172.21.15.111:0/2403984221 conn(0x55d456d8d4c0 msgr2=0x55d456d8d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c8be00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 0x7f7c700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 -- 172.21.15.111:0/2403984221 shutdown_connections 2024-04-24T22:47:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 -- 172.21.15.111:0/2403984221 wait complete. 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c94e4a740 1 -- start start 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.938+0000 7f7c8be00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 0x7f7c700a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c840049d0 tx=0x7f7c84009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 Processor -- start 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456d99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d456d9ed60 con 0x55d456d9c190 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d456cb0990 con 0x55d456d918d0 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d456d9eee0 con 0x55d456d90fb0 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c94e4a740 10 client.0 _ll_register_callbacks cb 0x55d456cb2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59836/0 (socket says 172.21.15.111:59836) 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c8be00640 1 -- 172.21.15.111:0/3326070831 learned_addr learned my addr 172.21.15.111:0/3326070831 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456d99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.939+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 -- 172.21.15.111:0/3326070831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 msgr2=0x55d456d991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 -- 172.21.15.111:0/3326070831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 msgr2=0x55d456d99c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 -- 172.21.15.111:0/3326070831 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d456ad5620 con 0x55d456d918d0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8be00640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456d99700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ccaf0 tx=0x7f7c78005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1fab4391cb4fd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c74c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801c070 con 0x55d456d918d0 2024-04-24T22:47:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f7c7809bb20 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c7c005520 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.940+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c780ba2e0 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 10 client.25709 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d456daad20 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c005520 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c78004bc0 con 0x55d456d918d0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 10 client.25709 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 20 client.25709 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456dac230 0x55d456dae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 4 client.25709 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d456d01ff0 con 0x55d456dac230 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.941+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456dac230 0x55d456dae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.942+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456dac230 0x55d456dae620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d09bd2960d21d9 server_cookie=b7ad81b0e187d9c3 in_seq=0 out_seq=0 2024-04-24T22:47:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.942+0000 7f7c74c00640 10 client.25709 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c74c00640 1 -- 172.21.15.111:0/3326070831 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d456d01ff0 con 0x55d456dac230 2024-04-24T22:47:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c74c00640 10 client.25709 handle_command_reply: tid=0 2024-04-24T22:47:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 client.25709 shutdown 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 msgr2=0x55d456d99700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456d99700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ccaf0 tx=0x7f7c78005290 comp rx=0 tx=0).stop 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 shutdown_connections 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456dac230 0x55d456dae620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d9c190 0x55d456d99c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456d99700 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c74c00640 0 client.25709 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3326070831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.943+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 >> 172.21.15.111:0/3326070831 conn(0x55d456d8d4c0 msgr2=0x55d456d931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.944+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 shutdown_connections 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.944+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3326070831 wait complete. 2024-04-24T22:47:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.944+0000 7f7c94e4a740 20 client.25709 trim_cache size 0 max 16384 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 Processor -- start 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 -- start start 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d8c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456db3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b1cb0 con 0x55d456d8d4c0 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1af580 con 0x55d456d918d0 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.960+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ad3d0 con 0x55d456d90fb0 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59844/0 (socket says 172.21.15.111:59844) 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 -- 172.21.15.111:0/3365487153 learned_addr learned my addr 172.21.15.111:0/3365487153 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d8c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8be00640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456db3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 -- 172.21.15.111:0/3365487153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 msgr2=0x55d456d8c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d8c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 -- 172.21.15.111:0/3365487153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 msgr2=0x55d456db3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456db3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8b400640 1 -- 172.21.15.111:0/3365487153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d456d8e650 con 0x55d456d8d4c0 2024-04-24T22:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.961+0000 7f7c8be00640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456db3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:47:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004b40 tx=0x7f7c7c004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ecb429ec9f5252 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c77400640 1 -- 172.21.15.111:0/3365487153 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c035040 con 0x55d456d8d4c0 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c77400640 1 -- 172.21.15.111:0/3365487153 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c004480 con 0x55d456d8d4c0 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c77400640 1 -- 172.21.15.111:0/3365487153 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0039b0 con 0x55d456d8d4c0 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3365487153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 msgr2=0x55d456d99a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004b40 tx=0x7f7c7c004b70 comp rx=0 tx=0).stop 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3365487153 shutdown_connections 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d918d0 0x55d456db3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d90fb0 0x55d456d8c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.962+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3365487153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456d99a60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.963+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3365487153 >> 172.21.15.111:0/3365487153 conn(0x55d456d88580 msgr2=0x55d456d8bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.963+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3365487153 shutdown_connections 2024-04-24T22:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.963+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3365487153 wait complete. 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.963+0000 7f7c94e4a740 1 -- start start 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.963+0000 7f7c94e4a740 1 Processor -- start 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 0x55d456da0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d918d0 0x55d456e13f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ada40 con 0x55d456d8d4c0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1aeeb0 con 0x55d456d90fb0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ae110 con 0x55d456d918d0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c94e4a740 10 client.0 _ll_register_callbacks cb 0x7f7c780053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59852/0 (socket says 172.21.15.111:59852) 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 -- 172.21.15.111:0/3839311168 learned_addr learned my addr 172.21.15.111:0/3839311168 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8be00640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d918d0 0x55d456e13f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 0x55d456da0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 -- 172.21.15.111:0/3839311168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d918d0 msgr2=0x55d456e13f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d918d0 0x55d456e13f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 -- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 msgr2=0x55d456da0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 0x55d456da0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.964+0000 7f7c8b400640 1 -- 172.21.15.111:0/3839311168 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d456e14570 con 0x55d456d8d4c0 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 0x55d456da0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c8b400640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a910 tx=0x7f7c7c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9affb21b260ff8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c74c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c035070 con 0x55d456d8d4c0 2024-04-24T22:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c7c004480 con 0x55d456d8d4c0 2024-04-24T22:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.965+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f7c7c0512e0 con 0x55d456d8d4c0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c062050 con 0x55d456d8d4c0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 10 client.15669 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d456e1b8f0 con 0x55d456d8d4c0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c004480 con 0x55d456d8d4c0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c7c0036f0 con 0x55d456d8d4c0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 10 client.15669 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 20 client.15669 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456d9c190 0x55d456e1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 4 client.15669 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.966+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d456cfc340 con 0x55d456d9c190 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.967+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456d9c190 0x55d456e1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.967+0000 7f7c8aa00640 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456d9c190 0x55d456e1db30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd436551d257a2be server_cookie=686541ca7df1575d in_seq=0 out_seq=0 2024-04-24T22:47:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.967+0000 7f7c74c00640 10 client.15669 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.982+0000 7f7c74c00640 1 -- 172.21.15.111:0/3839311168 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d456cfc340 con 0x55d456d9c190 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.982+0000 7f7c74c00640 10 client.15669 handle_command_reply: tid=0 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.982+0000 7f7c94e4a740 1 client.15669 shutdown 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.982+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 msgr2=0x55d456da0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.982+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a910 tx=0x7f7c7c005890 comp rx=0 tx=0).stop 2024-04-24T22:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 shutdown_connections 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55d456d9c190 0x55d456e1db30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d456d918d0 0x55d456e13f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c74c00640 0 client.15669 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d456d90fb0 0x55d456da0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 --2- 172.21.15.111:0/3839311168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d456d8d4c0 0x55d456da0410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 >> 172.21.15.111:0/3839311168 conn(0x55d456d88580 msgr2=0x55d456d8c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 shutdown_connections 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 1 -- 172.21.15.111:0/3839311168 wait complete. 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.983+0000 7f7c94e4a740 20 client.15669 trim_cache size 0 max 16384 2024-04-24T22:47:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 msgr2=0x7f7c700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 0x7f7c700a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c840049d0 tx=0x7f7c84009500 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 msgr2=0x7f7c8c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 0x7f7c8c19a5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 shutdown_connections 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c8c109760 0x7f7c8c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f7c700a2cf0 0x7f7c700a51b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c8c104110 0x7f7c8c19a5a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 --2- 172.21.15.111:0/977468200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c8c103750 0x7f7c8c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 >> 172.21.15.111:0/977468200 conn(0x7f7c8c0ff260 msgr2=0x7f7c8c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 shutdown_connections 2024-04-24T22:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:42.984+0000 7f7c90a00640 1 -- 172.21.15.111:0/977468200 wait complete. 2024-04-24T22:47:44.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:44 smithi171 ceph-mon[32337]: pgmap v176: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:47:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:44 smithi186 ceph-mon[32753]: pgmap v176: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:47:44.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:44 smithi111 ceph-mon[27746]: pgmap v176: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:47:46.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:46 smithi171 ceph-mon[32337]: pgmap v177: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:46 smithi186 ceph-mon[32753]: pgmap v177: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:46.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:46 smithi111 ceph-mon[27746]: pgmap v177: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:48 smithi171 ceph-mon[32337]: pgmap v178: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:48 smithi186 ceph-mon[32753]: pgmap v178: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:48.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:48 smithi111 ceph-mon[27746]: pgmap v178: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-24T22:47:50.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:50 smithi171 ceph-mon[32337]: pgmap v179: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:50.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:50 smithi186 ceph-mon[32753]: pgmap v179: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:50 smithi111 ceph-mon[27746]: pgmap v179: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-24T22:47:52.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:52 smithi171 ceph-mon[32337]: pgmap v180: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-24T22:47:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:52 smithi186 ceph-mon[32753]: pgmap v180: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-24T22:47:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:52 smithi111 ceph-mon[27746]: pgmap v180: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-24T22:47:54.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:54 smithi171 ceph-mon[32337]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-24T22:47:54.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:54 smithi186 ceph-mon[32753]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-24T22:47:54.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:54 smithi111 ceph-mon[27746]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 -- 172.21.15.111:0/1930693243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 msgr2=0x7f77e0076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 --2- 172.21.15.111:0/1930693243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e0076050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009900 tx=0x7f77d002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 -- 172.21.15.111:0/1930693243 shutdown_connections 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 --2- 172.21.15.111:0/1930693243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 0x7f77e01149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 --2- 172.21.15.111:0/1930693243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 0x7f77e0112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 --2- 172.21.15.111:0/1930693243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e0076050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.127+0000 7f77e5400640 1 -- 172.21.15.111:0/1930693243 >> 172.21.15.111:0/1930693243 conn(0x7f77e00ff440 msgr2=0x7f77e0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.128+0000 7f77e5400640 1 -- 172.21.15.111:0/1930693243 shutdown_connections 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.128+0000 7f77e5400640 1 -- 172.21.15.111:0/1930693243 wait complete. 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.128+0000 7f77e5400640 1 Processor -- start 2024-04-24T22:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 -- start start 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 0x7f77e019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 0x7f77e01a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0118170 con 0x7f77e0075c70 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0117ff0 con 0x7f77e01125d0 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77e5400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01182f0 con 0x7f77e0076590 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77de800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77de800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58410/0 (socket says 172.21.15.111:58410) 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77de800640 1 -- 172.21.15.111:0/2224319639 learned_addr learned my addr 172.21.15.111:0/2224319639 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.129+0000 7f77df200640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 0x7f77e01a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77dde00640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 0x7f77e019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 -- 172.21.15.111:0/2224319639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 msgr2=0x7f77e019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 0x7f77e019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 -- 172.21.15.111:0/2224319639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 msgr2=0x7f77e01a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 0x7f77e01a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 -- 172.21.15.111:0/2224319639 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e01a35c0 con 0x7f77e0075c70 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77de800640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77d002a9e0 tx=0x7f77d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77d7400640 1 -- 172.21.15.111:0/2224319639 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d003f070 con 0x7f77e0075c70 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77d7400640 1 -- 172.21.15.111:0/2224319639 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0031c30 con 0x7f77e0075c70 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77d7400640 1 -- 172.21.15.111:0/2224319639 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0035030 con 0x7f77e0075c70 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.130+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e01a3850 con 0x7f77e0075c70 2024-04-24T22:47:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.131+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e01ab140 con 0x7f77e0075c70 2024-04-24T22:47:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77d7400640 1 -- 172.21.15.111:0/2224319639 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f77d0042070 con 0x7f77e0075c70 2024-04-24T22:47:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 Processor -- start 2024-04-24T22:47:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 -- start start 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26d9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555f26cf0ab0 con 0x555f26d8ce00 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555f26c99640 con 0x555f26d97fe0 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77e827d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f26ceca70 con 0x555f26d8d720 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77de800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58422/0 (socket says 172.21.15.111:58422) 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 -- 172.21.15.111:0/224552677 learned_addr learned my addr 172.21.15.111:0/224552677 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77dde00640 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26d9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 -- 172.21.15.111:0/224552677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 msgr2=0x555f26d97aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d97aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 -- 172.21.15.111:0/224552677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 msgr2=0x555f26d9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26d9a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77df200640 1 -- 172.21.15.111:0/224552677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f26a05210 con 0x555f26d8ce00 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.135+0000 7f77d7400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 0x7f77c40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77df200640 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009f40 tx=0x7f77d80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bc86dc17854c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77d7400640 1 -- 172.21.15.111:0/2224319639 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f77d00efbb0 con 0x7f77e0075c70 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77cb400640 1 -- 172.21.15.111:0/224552677 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004d20 con 0x555f26d8ce00 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77de800640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 0x7f77c40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77cb400640 1 -- 172.21.15.111:0/224552677 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d8004050 con 0x555f26d8ce00 2024-04-24T22:47:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.136+0000 7f77cb400640 1 -- 172.21.15.111:0/224552677 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8005280 con 0x555f26d8ce00 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- 172.21.15.111:0/224552677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 msgr2=0x555f26d8d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009f40 tx=0x7f77d80118d0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77de800640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 0x7f77c40a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x555f26d8e3a0 tx=0x7f77d00f3660 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- 172.21.15.111:0/224552677 shutdown_connections 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26d9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d97aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 --2- 172.21.15.111:0/224552677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d8d1e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- 172.21.15.111:0/224552677 >> 172.21.15.111:0/224552677 conn(0x555f26d89310 msgr2=0x555f26d896f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- 172.21.15.111:0/224552677 shutdown_connections 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- 172.21.15.111:0/224552677 wait complete. 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.137+0000 7f77e827d740 1 -- start start 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 Processor -- start 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26d95900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555f26d9ac10 con 0x555f26d8ce00 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555f26cac990 con 0x555f26d8d720 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f26d9ad90 con 0x555f26d97fe0 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77e827d740 10 client.0 _ll_register_callbacks cb 0x555f26cae950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58424/0 (socket says 172.21.15.111:58424) 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77df200640 1 -- 172.21.15.111:0/3964319062 learned_addr learned my addr 172.21.15.111:0/3964319062 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.138+0000 7f77dde00640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26d95900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 -- 172.21.15.111:0/3964319062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 msgr2=0x555f26d95900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26d95900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 -- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 msgr2=0x555f26d953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 -- 172.21.15.111:0/3964319062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f26ad1620 con 0x555f26d8ce00 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77df200640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d8009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8383e476020abe8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004c00 con 0x555f26d8ce00 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f77d8004da0 con 0x555f26d8ce00 2024-04-24T22:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.139+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d8004050 con 0x555f26d8ce00 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f77d8036050 con 0x555f26d8ce00 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 10 client.15681 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f26da6aa0 con 0x555f26d8ce00 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d8004050 con 0x555f26d8ce00 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f77d8005160 con 0x555f26d8ce00 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 10 client.15681 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 20 client.15681 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26da7ea0 0x555f26daa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 4 client.15681 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.140+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f26cfdff0 con 0x555f26da7ea0 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.141+0000 7f77dde00640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26da7ea0 0x555f26daa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.141+0000 7f77dde00640 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26da7ea0 0x555f26daa290 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5043de6def38ee server_cookie=7e7741d8952cddb2 in_seq=0 out_seq=0 2024-04-24T22:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.141+0000 7f77bfe00640 10 client.15681 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77bfe00640 1 -- 172.21.15.111:0/3964319062 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f26cfdff0 con 0x555f26da7ea0 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77bfe00640 10 client.15681 handle_command_reply: tid=0 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 client.15681 shutdown 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 msgr2=0x555f26d94e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d8009c90 comp rx=0 tx=0).stop 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 shutdown_connections 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26da7ea0 0x555f26daa290 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26d95900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 --2- 172.21.15.111:0/3964319062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d94e80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77bfe00640 0 client.15681 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 >> 172.21.15.111:0/3964319062 conn(0x555f26d89310 msgr2=0x555f26d8f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.142+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 shutdown_connections 2024-04-24T22:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.143+0000 7f77e827d740 1 -- 172.21.15.111:0/3964319062 wait complete. 2024-04-24T22:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.143+0000 7f77e827d740 20 client.15681 trim_cache size 0 max 16384 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 Processor -- start 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 -- start start 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26dafa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b62b0 con 0x555f26d8ce00 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b3b80 con 0x555f26d8d720 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77e827d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b19d0 con 0x555f26d97fe0 2024-04-24T22:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58440/0 (socket says 172.21.15.111:58440) 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77df200640 1 -- 172.21.15.111:0/3897174650 learned_addr learned my addr 172.21.15.111:0/3897174650 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.159+0000 7f77dde00640 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26dafa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 -- 172.21.15.111:0/3897174650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 msgr2=0x555f26dafa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26dafa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 -- 172.21.15.111:0/3897174650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 msgr2=0x555f26d95720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d95720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 -- 172.21.15.111:0/3897174650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f26d89140 con 0x555f26d8ce00 2024-04-24T22:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77df200640 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009c60 tx=0x7f77d8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513edadd262bb4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77caa00640 1 -- 172.21.15.111:0/3897174650 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004100 con 0x555f26d8ce00 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77caa00640 1 -- 172.21.15.111:0/3897174650 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d802d070 con 0x555f26d8ce00 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.160+0000 7f77caa00640 1 -- 172.21.15.111:0/3897174650 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004cd0 con 0x555f26d8ce00 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 -- 172.21.15.111:0/3897174650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 msgr2=0x555f26d88c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009c60 tx=0x7f77d8014040 comp rx=0 tx=0).stop 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 -- 172.21.15.111:0/3897174650 shutdown_connections 2024-04-24T22:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d97fe0 0x555f26dafa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d8d720 0x555f26d95720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 --2- 172.21.15.111:0/3897174650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d88c00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 -- 172.21.15.111:0/3897174650 >> 172.21.15.111:0/3897174650 conn(0x555f26d89310 msgr2=0x555f26d8f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 -- 172.21.15.111:0/3897174650 shutdown_connections 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.161+0000 7f77e827d740 1 -- 172.21.15.111:0/3897174650 wait complete. 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 -- start start 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 Processor -- start 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26e0d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b2040 con 0x555f26d8ce00 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b34b0 con 0x555f26d97fe0 2024-04-24T22:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01b2710 con 0x555f26d8d720 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.162+0000 7f77e827d740 10 client.0 _ll_register_callbacks cb 0x7f77cc00d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58442/0 (socket says 172.21.15.111:58442) 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77df200640 1 -- 172.21.15.111:0/1614708651 learned_addr learned my addr 172.21.15.111:0/1614708651 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77df200640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26e0d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 -- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 msgr2=0x555f26e0d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26e0d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 -- 172.21.15.111:0/1614708651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 msgr2=0x555f26d921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 -- 172.21.15.111:0/1614708651 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f26e0d810 con 0x555f26d8d720 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77df200640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77dde00640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d926e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555f26d946e0 tx=0x7f77cc00de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0705fa2819b6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.163+0000 7f77bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01d580 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8469+0+0 (secure 0 0 0) 0x7f77cc01d720 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d802d070 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f77cc01d920 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 10 client.25765 fetch_fsmap learned FSMap version 24 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f26e0ea20 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d802d070 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f77cc004b80 con 0x555f26d8d720 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 10 client.25765 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 20 client.25765 populate_metadata read hostname 'smithi111' 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26e0f100 0x555f26e114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 4 client.25765 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.164+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f26cf8340 con 0x555f26e0f100 2024-04-24T22:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.165+0000 7f77df200640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26e0f100 0x555f26e114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.165+0000 7f77df200640 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26e0f100 0x555f26e114f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb19c13ee392f1e server_cookie=d6cdb7b4dc7c2625 in_seq=0 out_seq=0 2024-04-24T22:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.165+0000 7f77bfe00640 10 client.25765 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77bfe00640 1 -- 172.21.15.111:0/1614708651 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f26cf8340 con 0x555f26e0f100 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77bfe00640 10 client.25765 handle_command_reply: tid=0 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 client.25765 shutdown 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 msgr2=0x555f26d926e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d926e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555f26d946e0 tx=0x7f77cc00de80 comp rx=0 tx=0).stop 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 shutdown_connections 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x555f26e0f100 0x555f26e114f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f26d97fe0 0x555f26e0d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f26d8d720 0x555f26d926e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 --2- 172.21.15.111:0/1614708651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f26d8ce00 0x555f26d921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 >> 172.21.15.111:0/1614708651 conn(0x555f26d89310 msgr2=0x555f26d966b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77bfe00640 0 client.25765 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 shutdown_connections 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 1 -- 172.21.15.111:0/1614708651 wait complete. 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.179+0000 7f77e827d740 20 client.25765 trim_cache size 0 max 16384 2024-04-24T22:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 msgr2=0x7f77c40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 0x7f77c40a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x555f26d8e3a0 tx=0x7f77d00f3660 comp rx=0 tx=0).stop 2024-04-24T22:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 msgr2=0x7f77e019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77d002a9e0 tx=0x7f77d0003c10 comp rx=0 tx=0).stop 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 shutdown_connections 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77e01125d0 0x7f77e01a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f77c40a2e20 0x7f77c40a52e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e0076590 0x7f77e019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 --2- 172.21.15.111:0/2224319639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0075c70 0x7f77e019e5d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 >> 172.21.15.111:0/2224319639 conn(0x7f77e00ff440 msgr2=0x7f77e0114030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 shutdown_connections 2024-04-24T22:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:47:56.180+0000 7f77e5400640 1 -- 172.21.15.111:0/2224319639 wait complete. 2024-04-24T22:47:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:56 smithi171 ceph-mon[32337]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:47:56.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:56 smithi186 ceph-mon[32753]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:47:56.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:56 smithi111 ceph-mon[27746]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:47:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:47:58 smithi171 ceph-mon[32337]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:47:58.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:47:58 smithi186 ceph-mon[32753]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:47:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:47:58 smithi111 ceph-mon[27746]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:48:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:00 smithi171 ceph-mon[32337]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-04-24T22:48:00.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:00 smithi186 ceph-mon[32753]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-04-24T22:48:00.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:00 smithi111 ceph-mon[27746]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-04-24T22:48:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:02 smithi171 ceph-mon[32337]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-24T22:48:02.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:02 smithi186 ceph-mon[32753]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-24T22:48:02.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:02 smithi111 ceph-mon[27746]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-24T22:48:04.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:04 smithi171 ceph-mon[32337]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-04-24T22:48:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:04 smithi186 ceph-mon[32753]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-04-24T22:48:04.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:04 smithi111 ceph-mon[27746]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-04-24T22:48:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:06 smithi171 ceph-mon[32337]: pgmap v187: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 29 op/s 2024-04-24T22:48:06.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:06 smithi186 ceph-mon[32753]: pgmap v187: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 29 op/s 2024-04-24T22:48:06.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:06 smithi111 ceph-mon[27746]: pgmap v187: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 29 op/s 2024-04-24T22:48:07.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:07 smithi171 ceph-mon[32337]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T22:48:07.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:07 smithi171 ceph-mon[32337]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-24T22:48:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:07 smithi186 ceph-mon[32753]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T22:48:07.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:07 smithi186 ceph-mon[32753]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-24T22:48:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:07 smithi111 ceph-mon[27746]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T22:48:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:07 smithi111 ceph-mon[27746]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 -- 172.21.15.111:0/1988476780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 msgr2=0x7f079c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 --2- 172.21.15.111:0/1988476780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c109d30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f078c009ca0 tx=0x7f078c02a8c0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 -- 172.21.15.111:0/1988476780 shutdown_connections 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 --2- 172.21.15.111:0/1988476780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 0x7f079c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 --2- 172.21.15.111:0/1988476780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 --2- 172.21.15.111:0/1988476780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c109d30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.324+0000 7f07a1000640 1 -- 172.21.15.111:0/1988476780 >> 172.21.15.111:0/1988476780 conn(0x7f079c0ff470 msgr2=0x7f079c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.325+0000 7f07a1000640 1 -- 172.21.15.111:0/1988476780 shutdown_connections 2024-04-24T22:48:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.325+0000 7f07a1000640 1 -- 172.21.15.111:0/1988476780 wait complete. 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.326+0000 7f07a1000640 1 Processor -- start 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.326+0000 7f07a1000640 1 -- start start 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 0x7f079c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f079c113d30 con 0x7f079c103940 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f079c113bb0 con 0x7f079c109950 2024-04-24T22:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f07a1000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f079c113eb0 con 0x7f079c104300 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51028/0 (socket says 172.21.15.111:51028) 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 -- 172.21.15.111:0/2638815683 learned_addr learned my addr 172.21.15.111:0/2638815683 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f079a800640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0791400640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 0x7f079c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 -- 172.21.15.111:0/2638815683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 msgr2=0x7f079c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 0x7f079c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 -- 172.21.15.111:0/2638815683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 msgr2=0x7f079c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.327+0000 7f0799e00640 1 -- 172.21.15.111:0/2638815683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c19f1a0 con 0x7f079c103940 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f0799e00640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f0793400640 1 -- 172.21.15.111:0/2638815683 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c031450 con 0x7f079c103940 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f0793400640 1 -- 172.21.15.111:0/2638815683 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c004040 con 0x7f079c103940 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f0793400640 1 -- 172.21.15.111:0/2638815683 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c030970 con 0x7f079c103940 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c19f430 con 0x7f079c103940 2024-04-24T22:48:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.328+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c1a6d20 con 0x7f079c103940 2024-04-24T22:48:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.331+0000 7f07a3f7f740 1 Processor -- start 2024-04-24T22:48:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.331+0000 7f0793400640 1 -- 172.21.15.111:0/2638815683 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f078c0041f0 con 0x7f079c103940 2024-04-24T22:48:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 -- start start 2024-04-24T22:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 0x56162ee921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ee9f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56162edf5ab0 con 0x56162ee91e00 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56162ed9e640 con 0x56162ee92720 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56162edf1a70 con 0x56162ee9cfe0 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f0799e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.332+0000 7f079a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 0x56162ee921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:39066/0 (socket says 172.21.15.111:39066) 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0791400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ee9f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f079a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 0x56162ee921e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51038/0 (socket says 172.21.15.111:51038) 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 -- 172.21.15.111:0/1805551204 learned_addr learned my addr 172.21.15.111:0/1805551204 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 -- 172.21.15.111:0/1805551204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 msgr2=0x56162ee9f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ee9f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 -- 172.21.15.111:0/1805551204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 msgr2=0x56162ee921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 0x56162ee921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 -- 172.21.15.111:0/1805551204 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162eb0a210 con 0x56162ee92720 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0791400640 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ee9f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0793400640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 0x7f07800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0799e00640 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f078c0edc60 tx=0x7f078c0f65b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a087ae0dd59e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f079a800640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 0x7f07800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.333+0000 7f0793400640 1 -- 172.21.15.111:0/2638815683 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f078c0f0540 con 0x7f079c103940 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f078b400640 1 -- 172.21.15.111:0/1805551204 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0b8d90 con 0x56162ee92720 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f078b400640 1 -- 172.21.15.111:0/1805551204 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c036020 con 0x56162ee92720 2024-04-24T22:48:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f078b400640 1 -- 172.21.15.111:0/1805551204 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0bc450 con 0x56162ee92720 2024-04-24T22:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1805551204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 msgr2=0x56162ee9caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f079a800640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 0x7f07800a52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d20 tx=0x7f0794005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.334+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f078c0edc60 tx=0x7f078c0f65b0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1805551204 shutdown_connections 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ee9f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee92720 0x56162ee9caa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1805551204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee91e00 0x56162ee921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1805551204 >> 172.21.15.111:0/1805551204 conn(0x56162ee8e310 msgr2=0x56162ee8e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1805551204 shutdown_connections 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1805551204 wait complete. 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 -- start start 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.335+0000 7f07a3f7f740 1 Processor -- start 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee9a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56162ed9e640 con 0x56162ee9cfe0 2024-04-24T22:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56162edb1990 con 0x56162ee91e00 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56162ee9f9d0 con 0x56162ee92720 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f07a3f7f740 10 client.0 _ll_register_callbacks cb 0x56162edb3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f0799e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f0799e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:39072/0 (socket says 172.21.15.111:39072) 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f0799e00640 1 -- 172.21.15.111:0/2085709576 learned_addr learned my addr 172.21.15.111:0/2085709576 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f079a800640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.336+0000 7f0791400640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee9a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 -- 172.21.15.111:0/2085709576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 msgr2=0x56162ee9a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee9a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 -- 172.21.15.111:0/2085709576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 msgr2=0x56162ee9a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 -- 172.21.15.111:0/2085709576 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162ebd6620 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f0799e00640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f078c005e30 tx=0x7f078c005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a81651c804799f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f077fe00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0bc630 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f078c0bcd50 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c036020 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f078c0da050 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f07a3f7f740 10 client.25745 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56162eeab680 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.337+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c036020 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f078c0c52d0 con 0x56162ee91e00 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 10 client.25745 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 20 client.25745 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162eeaca70 0x56162eeaee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 4 client.25745 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162ee02ff0 con 0x56162eeaca70 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.338+0000 7f0791400640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162eeaca70 0x56162eeaee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.339+0000 7f0791400640 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162eeaca70 0x56162eeaee60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f43dddfe4287688 server_cookie=84163a9f8e6ee3a3 in_seq=0 out_seq=0 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.339+0000 7f077fe00640 10 client.25745 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.339+0000 7f077fe00640 1 -- 172.21.15.111:0/2085709576 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56162ee02ff0 con 0x56162eeaca70 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.339+0000 7f077fe00640 10 client.25745 handle_command_reply: tid=0 2024-04-24T22:48:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.339+0000 7f07a3f7f740 1 client.25745 shutdown 2024-04-24T22:48:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 msgr2=0x56162ee99ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f078c005e30 tx=0x7f078c005f60 comp rx=0 tx=0).stop 2024-04-24T22:48:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 shutdown_connections 2024-04-24T22:48:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162eeaca70 0x56162eeaee60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee9a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f077fe00640 0 client.25745 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/2085709576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162ee99ee0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 >> 172.21.15.111:0/2085709576 conn(0x56162ee8e310 msgr2=0x56162ee94090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 shutdown_connections 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 1 -- 172.21.15.111:0/2085709576 wait complete. 2024-04-24T22:48:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.340+0000 7f07a3f7f740 20 client.25745 trim_cache size 0 max 16384 2024-04-24T22:48:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 Processor -- start 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 -- start start 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee8d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b1f70 con 0x56162ee9cfe0 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1af840 con 0x56162ee91e00 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.356+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ad6b0 con 0x56162ee92720 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51056/0 (socket says 172.21.15.111:51056) 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 -- 172.21.15.111:0/1813907092 learned_addr learned my addr 172.21.15.111:0/1813907092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f0799e00640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f0791400640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee8d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 -- 172.21.15.111:0/1813907092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 msgr2=0x56162ee8d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee8d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 -- 172.21.15.111:0/1813907092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 msgr2=0x56162eea8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f079a800640 1 -- 172.21.15.111:0/1813907092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162ee98430 con 0x56162ee9cfe0 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.357+0000 7f0791400640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee8d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.358+0000 7f079a800640 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0794004350 tx=0x7f079401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a3df153487f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.358+0000 7f078aa00640 1 -- 172.21.15.111:0/1813907092 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079402e070 con 0x56162ee9cfe0 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.358+0000 7f078aa00640 1 -- 172.21.15.111:0/1813907092 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0794004380 con 0x56162ee9cfe0 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.358+0000 7f078aa00640 1 -- 172.21.15.111:0/1813907092 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794029820 con 0x56162ee9cfe0 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1813907092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 msgr2=0x56162ee9a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0794004350 tx=0x7f079401b000 comp rx=0 tx=0).stop 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1813907092 shutdown_connections 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee9cfe0 0x56162ee9a780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee92720 0x56162ee8d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/1813907092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1813907092 >> 172.21.15.111:0/1813907092 conn(0x56162ee8e310 msgr2=0x56162ee9b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1813907092 shutdown_connections 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- 172.21.15.111:0/1813907092 wait complete. 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 -- start start 2024-04-24T22:48:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.359+0000 7f07a3f7f740 1 Processor -- start 2024-04-24T22:48:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ef1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1add00 con 0x56162ee92720 2024-04-24T22:48:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1af170 con 0x56162ee91e00 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ae3d0 con 0x56162ee9cfe0 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f07a3f7f740 10 client.0 _ll_register_callbacks cb 0x7f0794009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f0791400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f0791400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51072/0 (socket says 172.21.15.111:51072) 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f0791400640 1 -- 172.21.15.111:0/844559207 learned_addr learned my addr 172.21.15.111:0/844559207 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f0799e00640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.360+0000 7f079a800640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ef1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 -- 172.21.15.111:0/844559207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 msgr2=0x56162ef1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ef1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 -- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 msgr2=0x56162eea0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 -- 172.21.15.111:0/844559207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162ef1aca0 con 0x56162ee92720 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0799e00640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f0791400640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0784004700 tx=0x7f0784005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07fa8849825ea28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f077fe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784020310 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f07840204b0 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.361+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0794004380 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0784030050 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 10 client.15705 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56162ef1bee0 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0794004380 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f078401e3f0 con 0x56162ee92720 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 10 client.15705 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 20 client.15705 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162ef1c690 0x56162ef1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 4 client.15705 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:48:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.362+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56162edfd340 con 0x56162ef1c690 2024-04-24T22:48:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.363+0000 7f0799e00640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162ef1c690 0x56162ef1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.363+0000 7f0799e00640 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162ef1c690 0x56162ef1eab0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7879d9252d901d4 server_cookie=d4906e44da309979 in_seq=0 out_seq=0 2024-04-24T22:48:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.363+0000 7f077fe00640 10 client.15705 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f077fe00640 1 -- 172.21.15.111:0/844559207 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56162edfd340 con 0x56162ef1c690 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f077fe00640 10 client.15705 handle_command_reply: tid=0 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 client.15705 shutdown 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 msgr2=0x56162eea0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0784004700 tx=0x7f0784005870 comp rx=0 tx=0).stop 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 shutdown_connections 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56162ef1c690 0x56162ef1eab0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56162ee9cfe0 0x56162ef1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162ee92720 0x56162eea0fb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 --2- 172.21.15.111:0/844559207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56162ee91e00 0x56162eea0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f077fe00640 0 client.15705 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 >> 172.21.15.111:0/844559207 conn(0x56162ee8e310 msgr2=0x56162ee9bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 shutdown_connections 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 1 -- 172.21.15.111:0/844559207 wait complete. 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.378+0000 7f07a3f7f740 20 client.15705 trim_cache size 0 max 16384 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:48:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:48:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:48:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:48:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 msgr2=0x7f07800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 0x7f07800a52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d20 tx=0x7f0794005f90 comp rx=0 tx=0).stop 2024-04-24T22:48:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 msgr2=0x7f079c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c003c10 comp rx=0 tx=0).stop 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 shutdown_connections 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f079c109950 0x7f079c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f07800a2e20 0x7f07800a52e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f079c104300 0x7f079c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 --2- 172.21.15.111:0/2638815683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f079c103940 0x7f079c19a1b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.379+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 >> 172.21.15.111:0/2638815683 conn(0x7f079c0ff470 msgr2=0x7f079c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.380+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 shutdown_connections 2024-04-24T22:48:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:09.380+0000 7f07a1000640 1 -- 172.21.15.111:0/2638815683 wait complete. 2024-04-24T22:48:09.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:09 smithi186 ceph-mon[32753]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:10.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:09 smithi111 ceph-mon[27746]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:10.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:09 smithi171 ceph-mon[32337]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:11.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:11 smithi186 ceph-mon[32753]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:12.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:11 smithi111 ceph-mon[27746]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:11 smithi171 ceph-mon[32337]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-04-24T22:48:13.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:13 smithi171 ceph-mon[32337]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T22:48:13.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:13 smithi186 ceph-mon[32753]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T22:48:13.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:13 smithi111 ceph-mon[27746]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T22:48:14.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:14 smithi171 ceph-mon[32337]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 19 op/s 2024-04-24T22:48:14.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:14 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:14 smithi186 ceph-mon[32753]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 19 op/s 2024-04-24T22:48:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:14 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:14 smithi111 ceph-mon[27746]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 19 op/s 2024-04-24T22:48:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:14 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:15.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:15 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:15.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:15 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:15.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:15 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:48:15.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:15 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:15.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:15 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:15.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:15 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:48:15.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:15 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:15.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:15 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:15.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:15 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:48:16.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:16 smithi171 ceph-mon[32337]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-24T22:48:16.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:16 smithi186 ceph-mon[32753]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-24T22:48:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:16 smithi111 ceph-mon[27746]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-24T22:48:18.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:18 smithi171 ceph-mon[32337]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:48:18.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:18 smithi186 ceph-mon[32753]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:48:18.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:18 smithi111 ceph-mon[27746]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:48:20.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:20 smithi171 ceph-mon[32337]: pgmap v196: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:20 smithi186 ceph-mon[32753]: pgmap v196: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:20 smithi111 ceph-mon[27746]: pgmap v196: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:22.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:22 smithi186 ceph-mon[32753]: pgmap v197: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 -- 172.21.15.111:0/3444291562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720107c80 msgr2=0x7fb7201080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 --2- 172.21.15.111:0/3444291562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720107c80 0x7fb7201080c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009860 tx=0x7fb70c02a340 comp rx=0 tx=0).stop 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 -- 172.21.15.111:0/3444291562 shutdown_connections 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 --2- 172.21.15.111:0/3444291562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720108600 0x7fb720114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 --2- 172.21.15.111:0/3444291562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720107c80 0x7fb7201080c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 --2- 172.21.15.111:0/3444291562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 0x7fb72010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 -- 172.21.15.111:0/3444291562 >> 172.21.15.111:0/3444291562 conn(0x7fb720077520 msgr2=0x7fb720106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 -- 172.21.15.111:0/3444291562 shutdown_connections 2024-04-24T22:48:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.521+0000 7fb725c00640 1 -- 172.21.15.111:0/3444291562 wait complete. 2024-04-24T22:48:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.522+0000 7fb725c00640 1 Processor -- start 2024-04-24T22:48:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.522+0000 7fb725c00640 1 -- start start 2024-04-24T22:48:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 0x7fb72019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 0x7fb7201a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb720117f80 con 0x7fb72010dc10 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb720117e00 con 0x7fb720108600 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb720118100 con 0x7fb720107c80 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb725200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 0x7fb7201a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb71fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb71f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 0x7fb72019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb71fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:35180/0 (socket says 172.21.15.111:35180) 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.523+0000 7fb71fe00640 1 -- 172.21.15.111:0/2372034060 learned_addr learned my addr 172.21.15.111:0/2372034060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 -- 172.21.15.111:0/2372034060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 msgr2=0x7fb72019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 0x7fb72019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 -- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 msgr2=0x7fb7201a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 0x7fb7201a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 -- 172.21.15.111:0/2372034060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7201a33e0 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71f400640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 0x7fb72019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71fe00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9c0 tx=0x7fb71000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.524+0000 7fb71cc00640 1 -- 172.21.15.111:0/2372034060 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71000ccc0 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.525+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7201a36d0 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.525+0000 7fb71cc00640 1 -- 172.21.15.111:0/2372034060 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb710005500 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.525+0000 7fb71cc00640 1 -- 172.21.15.111:0/2372034060 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710010430 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.525+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7201aaf60 con 0x7fb720107c80 2024-04-24T22:48:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 Processor -- start 2024-04-24T22:48:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 -- start start 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e1238d0 0x55959e12dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e130580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55959e086ab0 con 0x55959e122fb0 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55959e02f640 con 0x55959e1238d0 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.528+0000 7fb729d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55959e082a70 con 0x55959e12e190 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb71f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e130580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb71f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e130580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:35192/0 (socket says 172.21.15.111:35192) 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb71f400640 1 -- 172.21.15.111:0/120697123 learned_addr learned my addr 172.21.15.111:0/120697123 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47778/0 (socket says 172.21.15.111:47778) 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb71cc00640 1 -- 172.21.15.111:0/2372034060 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb7100046c0 con 0x7fb720107c80 2024-04-24T22:48:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 -- 172.21.15.111:0/120697123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 msgr2=0x55959e130580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e130580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 -- 172.21.15.111:0/120697123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e1238d0 msgr2=0x55959e12dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e1238d0 0x55959e12dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 -- 172.21.15.111:0/120697123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959dd9b210 con 0x55959e122fb0 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.529+0000 7fb725200640 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb718009f00 tx=0x7fb718011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bfa7eb545784a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb715600640 1 -- 172.21.15.111:0/120697123 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004d20 con 0x55959e122fb0 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb715600640 1 -- 172.21.15.111:0/120697123 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb718004030 con 0x55959e122fb0 2024-04-24T22:48:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb715600640 1 -- 172.21.15.111:0/120697123 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718005280 con 0x55959e122fb0 2024-04-24T22:48:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb71cc00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 0x7fb7040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb729d22740 1 -- 172.21.15.111:0/120697123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 msgr2=0x55959e123390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb729d22740 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb718009f00 tx=0x7fb718011810 comp rx=0 tx=0).stop 2024-04-24T22:48:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.530+0000 7fb729d22740 1 -- 172.21.15.111:0/120697123 shutdown_connections 2024-04-24T22:48:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e130580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e1238d0 0x55959e12dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 --2- 172.21.15.111:0/120697123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e123390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 -- 172.21.15.111:0/120697123 >> 172.21.15.111:0/120697123 conn(0x55959e11f4c0 msgr2=0x55959e11f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb71cc00640 1 -- 172.21.15.111:0/2372034060 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fb7100cf720 con 0x7fb720107c80 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb71fe00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 0x7fb7040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 -- 172.21.15.111:0/120697123 shutdown_connections 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 -- 172.21.15.111:0/120697123 wait complete. 2024-04-24T22:48:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb729d22740 1 -- start start 2024-04-24T22:48:22.535 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:22 smithi111 ceph-mon[27746]: pgmap v197: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.531+0000 7fb71fe00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 0x7fb7040a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55959e124550 tx=0x7fb710004a00 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 Processor -- start 2024-04-24T22:48:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e1238d0 0x55959e12b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e12bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55959e130dc0 con 0x55959e122fb0 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55959e042990 con 0x55959e12e190 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55959e130f40 con 0x55959e1238d0 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb729d22740 10 client.0 _ll_register_callbacks cb 0x55959e044950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb725200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb725200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47784/0 (socket says 172.21.15.111:47784) 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb725200640 1 -- 172.21.15.111:0/2005651044 learned_addr learned my addr 172.21.15.111:0/2005651044 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.532+0000 7fb71f400640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e1238d0 0x55959e12b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 -- 172.21.15.111:0/2005651044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e1238d0 msgr2=0x55959e12b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e1238d0 0x55959e12b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 -- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 msgr2=0x55959e12bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:48:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e12bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 -- 172.21.15.111:0/2005651044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959de67620 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb725200640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb718004460 tx=0x7fb718004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d8b89bdf1e93da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb70a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718027020 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fb718004c40 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.533+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb718004030 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb71801e040 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 10 client.15717 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55959e13cbf0 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb718004030 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb718018070 con 0x55959e122fb0 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 10 client.15717 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 20 client.15717 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e13dff0 0x55959e1403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 4 client.15717 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.534+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55959e093ff0 con 0x55959e13dff0 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.535+0000 7fb71f400640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e13dff0 0x55959e1403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.537+0000 7fb71f400640 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e13dff0 0x55959e1403e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0de2209cba3b781 server_cookie=300a8cf949312133 in_seq=0 out_seq=0 2024-04-24T22:48:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.537+0000 7fb70a000640 10 client.15717 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.537+0000 7fb70a000640 1 -- 172.21.15.111:0/2005651044 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55959e093ff0 con 0x55959e13dff0 2024-04-24T22:48:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.537+0000 7fb70a000640 10 client.15717 handle_command_reply: tid=0 2024-04-24T22:48:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.537+0000 7fb729d22740 1 client.15717 shutdown 2024-04-24T22:48:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 msgr2=0x55959e12b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb718004460 tx=0x7fb718004490 comp rx=0 tx=0).stop 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 shutdown_connections 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e13dff0 0x55959e1403e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e12bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e1238d0 0x55959e12b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 --2- 172.21.15.111:0/2005651044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e122fb0 0x55959e12b090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb70a000640 0 client.15717 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 >> 172.21.15.111:0/2005651044 conn(0x55959e11f4c0 msgr2=0x55959e125240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 shutdown_connections 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 1 -- 172.21.15.111:0/2005651044 wait complete. 2024-04-24T22:48:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.538+0000 7fb729d22740 20 client.15717 trim_cache size 0 max 16384 2024-04-24T22:48:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 Processor -- start 2024-04-24T22:48:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 -- start start 2024-04-24T22:48:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e122fb0 0x55959e125970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b6160 con 0x55959e1238d0 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b3a30 con 0x55959e122fb0 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.555+0000 7fb729d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b18a0 con 0x55959e12e190 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47800/0 (socket says 172.21.15.111:47800) 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 -- 172.21.15.111:0/1563603540 learned_addr learned my addr 172.21.15.111:0/1563603540 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb725200640 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e122fb0 0x55959e125970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb725200640 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 -- 172.21.15.111:0/1563603540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 msgr2=0x55959e146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 -- 172.21.15.111:0/1563603540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e122fb0 msgr2=0x55959e125970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e122fb0 0x55959e125970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 -- 172.21.15.111:0/1563603540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959e13a720 con 0x55959e1238d0 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.556+0000 7fb71f400640 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009830 tx=0x7fb70c02aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a19c69ffc32315 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.557+0000 7fb714c00640 1 -- 172.21.15.111:0/1563603540 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c037040 con 0x55959e1238d0 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.557+0000 7fb714c00640 1 -- 172.21.15.111:0/1563603540 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb70c0309e0 con 0x55959e1238d0 2024-04-24T22:48:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.557+0000 7fb714c00640 1 -- 172.21.15.111:0/1563603540 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c032440 con 0x55959e1238d0 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.557+0000 7fb729d22740 1 -- 172.21.15.111:0/1563603540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 msgr2=0x55959e145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.557+0000 7fb729d22740 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009830 tx=0x7fb70c02aad0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 -- 172.21.15.111:0/1563603540 shutdown_connections 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e12e190 0x55959e146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e145b00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 --2- 172.21.15.111:0/1563603540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e122fb0 0x55959e125970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 -- 172.21.15.111:0/1563603540 >> 172.21.15.111:0/1563603540 conn(0x55959e11f4c0 msgr2=0x55959e138c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 -- 172.21.15.111:0/1563603540 shutdown_connections 2024-04-24T22:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 -- 172.21.15.111:0/1563603540 wait complete. 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 -- start start 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.558+0000 7fb729d22740 1 Processor -- start 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e133890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b1ef0 con 0x55959e1238d0 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b3360 con 0x55959e12e190 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b25c0 con 0x55959e122fb0 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb729d22740 10 client.0 _ll_register_callbacks cb 0x7fb710005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb71f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:35220/0 (socket says 172.21.15.111:35220) 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 -- 172.21.15.111:0/104195950 learned_addr learned my addr 172.21.15.111:0/104195950 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 -- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 msgr2=0x55959e133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.559+0000 7fb725200640 1 -- 172.21.15.111:0/104195950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 msgr2=0x55959e133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb725200640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb725200640 1 -- 172.21.15.111:0/104195950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959e1345e0 con 0x55959e122fb0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb725200640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb718009270 tx=0x7fb718003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a7d70d198ffd898 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb70a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718027040 con 0x55959e122fb0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fb71800f830 con 0x55959e122fb0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.560+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55959e08e340 con 0x55959e122fb0 2024-04-24T22:48:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb71800fa30 con 0x55959e122fb0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 10 client.25801 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55959e1a4360 con 0x55959e122fb0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55959e08e340 con 0x55959e122fb0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb718018070 con 0x55959e122fb0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 10 client.25801 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 20 client.25801 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e1a4a20 0x55959e1a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 4 client.25801 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.561+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55959e00c7a0 con 0x55959e1a4a20 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.562+0000 7fb71f400640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e1a4a20 0x55959e1a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.562+0000 7fb71f400640 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e1a4a20 0x55959e1a6e40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d249d75693fb5f server_cookie=ab0828029cd794d9 in_seq=0 out_seq=0 2024-04-24T22:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.562+0000 7fb70a000640 10 client.25801 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.576+0000 7fb70a000640 1 -- 172.21.15.111:0/104195950 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55959e00c7a0 con 0x55959e1a4a20 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.576+0000 7fb70a000640 10 client.25801 handle_command_reply: tid=0 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.576+0000 7fb729d22740 1 client.25801 shutdown 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.576+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 msgr2=0x55959e136550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.576+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb718009270 tx=0x7fb718003f80 comp rx=0 tx=0).stop 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 shutdown_connections 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55959e1a4a20 0x55959e1a6e40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55959e12e190 0x55959e133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55959e1238d0 0x55959e133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 --2- 172.21.15.111:0/104195950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55959e122fb0 0x55959e136550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb70a000640 0 client.25801 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 >> 172.21.15.111:0/104195950 conn(0x55959e11f4c0 msgr2=0x55959e139500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 shutdown_connections 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 1 -- 172.21.15.111:0/104195950 wait complete. 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.577+0000 7fb729d22740 20 client.25801 trim_cache size 0 max 16384 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 msgr2=0x7fb7040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 0x7fb7040a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55959e124550 tx=0x7fb710004a00 comp rx=0 tx=0).stop 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 msgr2=0x7fb72019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9c0 tx=0x7fb71000ee80 comp rx=0 tx=0).stop 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 shutdown_connections 2024-04-24T22:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72010dc10 0x7fb7201a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb720108600 0x7fb72019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7040a2cf0 0x7fb7040a51b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 --2- 172.21.15.111:0/2372034060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb720107c80 0x7fb72019e440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 >> 172.21.15.111:0/2372034060 conn(0x7fb720077520 msgr2=0x7fb7200785d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 shutdown_connections 2024-04-24T22:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:22.578+0000 7fb725c00640 1 -- 172.21.15.111:0/2372034060 wait complete. 2024-04-24T22:48:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:22 smithi171 ceph-mon[32337]: pgmap v197: 97 pgs: 97 active+clean; 2.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-04-24T22:48:24.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:24 smithi186 ceph-mon[32753]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 68 KiB/s wr, 21 op/s 2024-04-24T22:48:24.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:24 smithi111 ceph-mon[27746]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 68 KiB/s wr, 21 op/s 2024-04-24T22:48:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:24 smithi171 ceph-mon[32337]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 68 KiB/s wr, 21 op/s 2024-04-24T22:48:26.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:26 smithi186 ceph-mon[32753]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:26.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:26 smithi111 ceph-mon[27746]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:26 smithi171 ceph-mon[32337]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:28.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:28 smithi186 ceph-mon[32753]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:28.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:28 smithi111 ceph-mon[27746]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:28 smithi171 ceph-mon[32337]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-24T22:48:30.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:30 smithi186 ceph-mon[32753]: pgmap v201: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:30.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:30 smithi111 ceph-mon[27746]: pgmap v201: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:30 smithi171 ceph-mon[32337]: pgmap v201: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:32.459 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:32 smithi171 ceph-mon[32337]: pgmap v202: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:32.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:32 smithi186 ceph-mon[32753]: pgmap v202: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:32.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:32 smithi111 ceph-mon[27746]: pgmap v202: 97 pgs: 97 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:251/600: open fd = 3 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.208 INFO:tasks.workunit.client.0.smithi111.stdout:252/600: open fd = 3 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:253/600: open fd = 3 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:254/600: open fd = 3 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:255/600: open fd = 3 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.209 INFO:tasks.workunit.client.0.smithi111.stdout:256/600: open fd = 3 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:257/600: open fd = 3 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:258/600: open fd = 3 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:259/600: open fd = 3 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.210 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:260/600: open fd = 3 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:261/600: open fd = 3 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:262/600: open fd = 3 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.211 INFO:tasks.workunit.client.0.smithi111.stdout:263/600: open fd = 3 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:264/600: open fd = 3 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:265/600: open fd = 3 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:266/600: open fd = 3 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.212 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:267/600: open fd = 3 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:268/600: open fd = 3 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:269/600: open fd = 3 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:270/600: open fd = 3 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.213 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:271/600: open fd = 3 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:272/600: open fd = 3 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:273/600: open fd = 3 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.214 INFO:tasks.workunit.client.0.smithi111.stdout:274/600: open fd = 3 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:275/600: open fd = 3 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:276/600: open fd = 3 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:277/600: open fd = 3 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.215 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:278/600: open fd = 3 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:279/600: open fd = 3 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:280/600: open fd = 3 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:281/600: open fd = 3 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.216 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:282/600: open fd = 3 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:283/600: open fd = 3 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:284/600: open fd = 3 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:285/600: open fd = 3 2024-04-24T22:48:34.217 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:286/600: open fd = 3 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:287/600: open fd = 3 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:288/600: open fd = 3 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.218 INFO:tasks.workunit.client.0.smithi111.stdout:289/600: open fd = 3 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:290/600: open fd = 3 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:291/600: open fd = 3 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:292/600: open fd = 3 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.219 INFO:tasks.workunit.client.0.smithi111.stdout:293/600: open fd = 3 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:294/600: open fd = 3 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:295/600: open fd = 3 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:296/600: open fd = 3 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.220 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:297/600: open fd = 3 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:298/600: open fd = 3 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:299/600: open fd = 3 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:300/600: open fd = 3 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.221 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:301/600: open fd = 3 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:302/600: open fd = 3 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:303/600: open fd = 3 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:304/600: open fd = 3 2024-04-24T22:48:34.222 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:305/600: open fd = 3 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:306/600: open fd = 3 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:307/600: open fd = 3 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.223 INFO:tasks.workunit.client.0.smithi111.stdout:308/600: open fd = 3 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:309/600: open fd = 3 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:310/600: open fd = 3 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:311/600: open fd = 3 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.224 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.225 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:48:34.225 INFO:tasks.workunit.client.0.smithi111.stdout:312/600: open fd = 3 2024-04-24T22:48:34.225 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.225 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:48:34.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:34 smithi186 ceph-mon[32753]: pgmap v203: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:34.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:34 smithi111 ceph-mon[27746]: pgmap v203: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:34 smithi171 ceph-mon[32337]: pgmap v203: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 -- 172.21.15.111:0/3734919204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed8075c70 msgr2=0x7f5ed8076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 --2- 172.21.15.111:0/3734919204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed8075c70 0x7f5ed8076050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009900 tx=0x7f5ec802a410 comp rx=0 tx=0).stop 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 -- 172.21.15.111:0/3734919204 shutdown_connections 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 --2- 172.21.15.111:0/3734919204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed81125e0 0x7f5ed81149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 --2- 172.21.15.111:0/3734919204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8076590 0x7f5ed81120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 --2- 172.21.15.111:0/3734919204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed8075c70 0x7f5ed8076050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.720+0000 7f5edca00640 1 -- 172.21.15.111:0/3734919204 >> 172.21.15.111:0/3734919204 conn(0x7f5ed80ff470 msgr2=0x7f5ed8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.721+0000 7f5edca00640 1 -- 172.21.15.111:0/3734919204 shutdown_connections 2024-04-24T22:48:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.721+0000 7f5edca00640 1 -- 172.21.15.111:0/3734919204 wait complete. 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 Processor -- start 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 -- start start 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8075c70 0x7f5ed819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed8076590 0x7f5ed819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8118180 con 0x7f5ed81125e0 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8118000 con 0x7f5ed8075c70 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5edca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8118300 con 0x7f5ed8076590 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5ed6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5ed6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53824/0 (socket says 172.21.15.111:53824) 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.722+0000 7f5ed6800640 1 -- 172.21.15.111:0/379583204 learned_addr learned my addr 172.21.15.111:0/379583204 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed8076590 0x7f5ed819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8075c70 0x7f5ed819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 -- 172.21.15.111:0/379583204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed8076590 msgr2=0x7f5ed819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed8076590 0x7f5ed819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 -- 172.21.15.111:0/379583204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8075c70 msgr2=0x7f5ed819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8075c70 0x7f5ed819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 -- 172.21.15.111:0/379583204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed81a35b0 con 0x7f5ed81125e0 2024-04-24T22:48:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ed6800640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000eb40 tx=0x7f5ed000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ecea00640 1 -- 172.21.15.111:0/379583204 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0004b20 con 0x7f5ed81125e0 2024-04-24T22:48:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5ecea00640 1 -- 172.21.15.111:0/379583204 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ed0004cc0 con 0x7f5ed81125e0 2024-04-24T22:48:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.723+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed81a38a0 con 0x7f5ed81125e0 2024-04-24T22:48:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.724+0000 7f5ecea00640 1 -- 172.21.15.111:0/379583204 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0010850 con 0x7f5ed81125e0 2024-04-24T22:48:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.724+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed81ab130 con 0x7f5ed81125e0 2024-04-24T22:48:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 Processor -- start 2024-04-24T22:48:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5ecea00640 1 -- 172.21.15.111:0/379583204 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5ed0005020 con 0x7f5ed81125e0 2024-04-24T22:48:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 -- start start 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c2a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561a2c1feab0 con 0x561a2c2a5fe0 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a2c1a7640 con 0x561a2c29ae00 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.727+0000 7f5edf808740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561a2c1faa70 con 0x561a2c29b720 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c2a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:46666/0 (socket says 172.21.15.111:46666) 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 -- 172.21.15.111:0/467087201 learned_addr learned my addr 172.21.15.111:0/467087201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 -- 172.21.15.111:0/467087201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 msgr2=0x561a2c2a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c2a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 -- 172.21.15.111:0/467087201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 msgr2=0x561a2c2a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecfe00640 1 -- 172.21.15.111:0/467087201 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2bf13210 con 0x561a2c29ae00 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.728+0000 7f5ecea00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 0x7f5ebc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009fd0 tx=0x7f5ec0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7b32d8b0a334a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ecea00640 1 -- 172.21.15.111:0/379583204 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f5ed00cfc10 con 0x7f5ed81125e0 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 0x7f5ebc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ec7000640 1 -- 172.21.15.111:0/467087201 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004030 con 0x561a2c29ae00 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ec7000640 1 -- 172.21.15.111:0/467087201 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0005500 con 0x561a2c29ae00 2024-04-24T22:48:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.729+0000 7f5ec7000640 1 -- 172.21.15.111:0/467087201 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004ca0 con 0x561a2c29ae00 2024-04-24T22:48:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- 172.21.15.111:0/467087201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 msgr2=0x561a2c29b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009fd0 tx=0x7f5ec0005850 comp rx=0 tx=0).stop 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 0x7f5ebc0a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8000c00 tx=0x7f5ec8005b50 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- 172.21.15.111:0/467087201 shutdown_connections 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c2a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 --2- 172.21.15.111:0/467087201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c29b1e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- 172.21.15.111:0/467087201 >> 172.21.15.111:0/467087201 conn(0x561a2c297310 msgr2=0x561a2c2976f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- 172.21.15.111:0/467087201 shutdown_connections 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- 172.21.15.111:0/467087201 wait complete. 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.730+0000 7f5edf808740 1 -- start start 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 Processor -- start 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561a2c2a8c10 con 0x561a2c29b720 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a2c1ba990 con 0x561a2c29ae00 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561a2c2a8d90 con 0x561a2c2a5fe0 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5edf808740 10 client.0 _ll_register_callbacks cb 0x561a2c1bc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53830/0 (socket says 172.21.15.111:53830) 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5ecfe00640 1 -- 172.21.15.111:0/1372880672 learned_addr learned my addr 172.21.15.111:0/1372880672 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.731+0000 7f5ed6800640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 -- 172.21.15.111:0/1372880672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 msgr2=0x561a2c2a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 -- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 msgr2=0x561a2c2a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 -- 172.21.15.111:0/1372880672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2bfdf620 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ed6800640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0004620 tx=0x7f5ec0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3fd56612a93ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.732+0000 7f5ebb400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec002c060 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0005500 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f5ec0004df0 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ec000f790 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561a2c2b4ab0 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0005500 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ec0026030 con 0x561a2c29b720 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 20 client.15735 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.733+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c2b5e00 0x561a2c2b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.734+0000 7f5edf808740 4 client.15735 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:48:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.734+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2c20bff0 con 0x561a2c2b5e00 2024-04-24T22:48:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.734+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c2b5e00 0x561a2c2b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.734+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c2b5e00 0x561a2c2b81f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59130b127e663c7 server_cookie=42af5623ffc49d68 in_seq=0 out_seq=0 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.734+0000 7f5ebb400640 10 client.15735 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5ebb400640 1 -- 172.21.15.111:0/1372880672 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a2c20bff0 con 0x561a2c2b5e00 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5ebb400640 10 client.15735 handle_command_reply: tid=0 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5edf808740 1 client.15735 shutdown 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 msgr2=0x561a2c2a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0004620 tx=0x7f5ec0009da0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.735+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 shutdown_connections 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c2b5e00 0x561a2c2b81f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c2a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29b720 0x561a2c2a3490 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 --2- 172.21.15.111:0/1372880672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5ebb400640 0 client.15735 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 >> 172.21.15.111:0/1372880672 conn(0x561a2c297310 msgr2=0x561a2c29d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 shutdown_connections 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 1 -- 172.21.15.111:0/1372880672 wait complete. 2024-04-24T22:48:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.736+0000 7f5edf808740 20 client.15735 trim_cache size 0 max 16384 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 Processor -- start 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 -- start start 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2bd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b62e0 con 0x561a2c2a5fe0 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b3bb0 con 0x561a2c29ae00 2024-04-24T22:48:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.752+0000 7f5edf808740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b1a20 con 0x561a2c29b720 2024-04-24T22:48:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ed6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2bd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:33898/0 (socket says 172.21.15.111:33898) 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 -- 172.21.15.111:0/2552567279 learned_addr learned my addr 172.21.15.111:0/2552567279 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 -- 172.21.15.111:0/2552567279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 msgr2=0x561a2c2b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 -- 172.21.15.111:0/2552567279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 msgr2=0x561a2c2bd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2bd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 -- 172.21.15.111:0/2552567279 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2c2b8520 con 0x561a2c29b720 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.753+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0004990 tx=0x7f5ec0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f7133f0a44dc8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5ec6600640 1 -- 172.21.15.111:0/2552567279 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec002c070 con 0x561a2c29b720 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5ec6600640 1 -- 172.21.15.111:0/2552567279 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0009a90 con 0x561a2c29b720 2024-04-24T22:48:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5ec6600640 1 -- 172.21.15.111:0/2552567279 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000f930 con 0x561a2c29b720 2024-04-24T22:48:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 -- 172.21.15.111:0/2552567279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 msgr2=0x561a2c296300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0004990 tx=0x7f5ec0011720 comp rx=0 tx=0).stop 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 -- 172.21.15.111:0/2552567279 shutdown_connections 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c2a5fe0 0x561a2c2bd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c29b720 0x561a2c296300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 --2- 172.21.15.111:0/2552567279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29ae00 0x561a2c2b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.754+0000 7f5edf808740 1 -- 172.21.15.111:0/2552567279 >> 172.21.15.111:0/2552567279 conn(0x561a2c297310 msgr2=0x561a2c295bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.755+0000 7f5edf808740 1 -- 172.21.15.111:0/2552567279 shutdown_connections 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.755+0000 7f5edf808740 1 -- 172.21.15.111:0/2552567279 wait complete. 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.755+0000 7f5edf808740 1 -- start start 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.755+0000 7f5edf808740 1 Processor -- start 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 0x561a2c2ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c29ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b2070 con 0x561a2c29ae00 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b34e0 con 0x561a2c29b720 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b2740 con 0x561a2c2a5fe0 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5edf808740 10 client.0 _ll_register_callbacks cb 0x7f5ec0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 0x561a2c2ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:46696/0 (socket says 172.21.15.111:46696) 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 -- 172.21.15.111:0/4164431304 learned_addr learned my addr 172.21.15.111:0/4164431304 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ed6800640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c29ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 -- 172.21.15.111:0/4164431304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 msgr2=0x561a2c29ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c29ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 -- 172.21.15.111:0/4164431304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 msgr2=0x561a2c2ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 0x561a2c2ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ecfe00640 1 -- 172.21.15.111:0/4164431304 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2c2a0740 con 0x561a2c29b720 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.756+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 0x561a2c2ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5ecfe00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000a010 tx=0x7f5ec00114c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5617a3defbb5718 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5ebb400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:48:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec003d020 con 0x561a2c29b720 2024-04-24T22:48:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f5ec0004040 con 0x561a2c29b720 2024-04-24T22:48:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0009a90 con 0x561a2c29b720 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ec0048050 con 0x561a2c29b720 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5edf808740 10 client.25781 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.757+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561a2c31c290 con 0x561a2c29b720 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0009a90 con 0x561a2c29b720 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ec0026070 con 0x561a2c29b720 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 10 client.25781 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 20 client.25781 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c31c930 0x561a2c31ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 4 client.25781 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a2c206340 con 0x561a2c31c930 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.758+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c31c930 0x561a2c31ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.759+0000 7f5ed5e00640 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c31c930 0x561a2c31ed50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51e7460505d7a4f server_cookie=dc0ebf85a84860f3 in_seq=0 out_seq=0 2024-04-24T22:48:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.759+0000 7f5ebb400640 10 client.25781 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5ebb400640 1 -- 172.21.15.111:0/4164431304 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a2c206340 con 0x561a2c31c930 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5ebb400640 10 client.25781 handle_command_reply: tid=0 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 client.25781 shutdown 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 msgr2=0x561a2c29f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000a010 tx=0x7f5ec00114c0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 shutdown_connections 2024-04-24T22:48:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561a2c31c930 0x561a2c31ed50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a2c2a5fe0 0x561a2c29ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a2c29b720 0x561a2c29f9b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 --2- 172.21.15.111:0/4164431304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561a2c29ae00 0x561a2c2ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5ebb400640 0 client.25781 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 >> 172.21.15.111:0/4164431304 conn(0x561a2c297310 msgr2=0x561a2c296480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 shutdown_connections 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.773+0000 7f5edf808740 1 -- 172.21.15.111:0/4164431304 wait complete. 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edf808740 20 client.25781 trim_cache size 0 max 16384 2024-04-24T22:48:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:48:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:48:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:48:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:48:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 msgr2=0x7f5ebc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 0x7f5ebc0a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8000c00 tx=0x7f5ec8005b50 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 msgr2=0x7f5ed81a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000eb40 tx=0x7f5ed000c2d0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 shutdown_connections 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ed81125e0 0x7f5ed81a2e80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f5ebc0a2da0 0x7f5ebc0a5260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ed8076590 0x7f5ed819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 --2- 172.21.15.111:0/379583204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ed8075c70 0x7f5ed819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.774+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 >> 172.21.15.111:0/379583204 conn(0x7f5ed80ff470 msgr2=0x7f5ed8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.775+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 shutdown_connections 2024-04-24T22:48:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:35.775+0000 7f5edca00640 1 -- 172.21.15.111:0/379583204 wait complete. 2024-04-24T22:48:36.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:36 smithi186 ceph-mon[32753]: pgmap v204: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:36.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:36 smithi111 ceph-mon[27746]: pgmap v204: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:36.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:36 smithi171 ceph-mon[32337]: pgmap v204: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:38.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:38 smithi186 ceph-mon[32753]: pgmap v205: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:38.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:38 smithi111 ceph-mon[27746]: pgmap v205: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:38 smithi171 ceph-mon[32337]: pgmap v205: 97 pgs: 97 active+clean; 3.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-04-24T22:48:40.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:40 smithi186 ceph-mon[32753]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 19 op/s 2024-04-24T22:48:40.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:40 smithi111 ceph-mon[27746]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 19 op/s 2024-04-24T22:48:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:40 smithi171 ceph-mon[32337]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 19 op/s 2024-04-24T22:48:42.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:42 smithi186 ceph-mon[32753]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:42.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:42 smithi111 ceph-mon[27746]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:42 smithi171 ceph-mon[32337]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:44 smithi186 ceph-mon[32753]: pgmap v208: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:44.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:44 smithi111 ceph-mon[27746]: pgmap v208: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:44.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:44 smithi171 ceph-mon[32337]: pgmap v208: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:46.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:46 smithi186 ceph-mon[32753]: pgmap v209: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:46.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:46 smithi111 ceph-mon[27746]: pgmap v209: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:46.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:46 smithi171 ceph-mon[32337]: pgmap v209: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:48 smithi186 ceph-mon[32753]: pgmap v210: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:48.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:48 smithi111 ceph-mon[27746]: pgmap v210: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:48 smithi171 ceph-mon[32337]: pgmap v210: 97 pgs: 97 active+clean; 3.6 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:48:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 -- 172.21.15.111:0/2587515532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 msgr2=0x7febcc110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 --2- 172.21.15.111:0/2587515532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc110810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7febc4009c10 tx=0x7febc402a6e0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 -- 172.21.15.111:0/2587515532 shutdown_connections 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 --2- 172.21.15.111:0/2587515532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc110810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 --2- 172.21.15.111:0/2587515532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 --2- 172.21.15.111:0/2587515532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 0x7febcc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 -- 172.21.15.111:0/2587515532 >> 172.21.15.111:0/2587515532 conn(0x7febcc0ff410 msgr2=0x7febcc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 -- 172.21.15.111:0/2587515532 shutdown_connections 2024-04-24T22:48:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.918+0000 7febd4200640 1 -- 172.21.15.111:0/2587515532 wait complete. 2024-04-24T22:48:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 Processor -- start 2024-04-24T22:48:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 -- start start 2024-04-24T22:48:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 0x7febcc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7febcc113cd0 con 0x7febcc109950 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febcc113b50 con 0x7febcc103940 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.920+0000 7febd4200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febcc113e50 con 0x7febcc104300 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 0x7febcc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43814/0 (socket says 172.21.15.111:43814) 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 -- 172.21.15.111:0/3764991489 learned_addr learned my addr 172.21.15.111:0/3764991489 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 -- 172.21.15.111:0/3764991489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 msgr2=0x7febcc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 -- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 msgr2=0x7febcc19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 0x7febcc19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd1800640 1 -- 172.21.15.111:0/3764991489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc19f1c0 con 0x7febcc103940 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.921+0000 7febd0e00640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febd1800640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9c0 tx=0x7febc000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febca000640 1 -- 172.21.15.111:0/3764991489 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000ccc0 con 0x7febcc103940 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc19f4b0 con 0x7febcc103940 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febca000640 1 -- 172.21.15.111:0/3764991489 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc0005500 con 0x7febcc103940 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febca000640 1 -- 172.21.15.111:0/3764991489 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0010430 con 0x7febcc103940 2024-04-24T22:48:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.922+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1a6d40 con 0x7febcc103940 2024-04-24T22:48:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.925+0000 7febd707f740 1 Processor -- start 2024-04-24T22:48:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.925+0000 7febd707f740 1 -- start start 2024-04-24T22:48:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.925+0000 7febca000640 1 -- 172.21.15.111:0/3764991489 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7febc00046c0 con 0x7febcc103940 2024-04-24T22:48:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5628f8846ab0 con 0x5628f88edfe0 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5628f87ef640 con 0x5628f88e3720 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd707f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5628f8842a70 con 0x5628f88e2e00 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60924/0 (socket says 172.21.15.111:60924) 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 -- 172.21.15.111:0/4003535414 learned_addr learned my addr 172.21.15.111:0/4003535414 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 -- 172.21.15.111:0/4003535414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 msgr2=0x5628f88e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.926+0000 7febd0e00640 1 -- 172.21.15.111:0/4003535414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 msgr2=0x5628f88edaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febd0e00640 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febd0e00640 1 -- 172.21.15.111:0/4003535414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f855b210 con 0x5628f88edfe0 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febca000640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febd0e00640 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febb80114f0 tx=0x7febb80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bd3f531e1f87dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febca000640 1 -- 172.21.15.111:0/3764991489 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7febc00cf6f0 con 0x7febcc103940 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febd2200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febbea00640 1 -- 172.21.15.111:0/4003535414 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb801e070 con 0x5628f88edfe0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febbea00640 1 -- 172.21.15.111:0/4003535414 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb800faa0 con 0x5628f88edfe0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.927+0000 7febbea00640 1 -- 172.21.15.111:0/4003535414 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8019c40 con 0x5628f88edfe0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 -- 172.21.15.111:0/4003535414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 msgr2=0x5628f88f03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd2200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 0x7febb40a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febc4004af0 tx=0x7febc4005e30 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febb80114f0 tx=0x7febb80119b0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 -- 172.21.15.111:0/4003535414 shutdown_connections 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88f03d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 --2- 172.21.15.111:0/4003535414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 -- 172.21.15.111:0/4003535414 >> 172.21.15.111:0/4003535414 conn(0x5628f88df310 msgr2=0x5628f88df6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 -- 172.21.15.111:0/4003535414 shutdown_connections 2024-04-24T22:48:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.928+0000 7febd707f740 1 -- 172.21.15.111:0/4003535414 wait complete. 2024-04-24T22:48:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.929+0000 7febd707f740 1 -- start start 2024-04-24T22:48:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.929+0000 7febd707f740 1 Processor -- start 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.929+0000 7febd707f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.929+0000 7febd707f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd707f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd707f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5628f88f0c10 con 0x5628f88edfe0 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd707f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5628f8802990 con 0x5628f88e3720 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd707f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5628f88f0d90 con 0x5628f88e2e00 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd707f740 10 client.0 _ll_register_callbacks cb 0x5628f8804950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:59202/0 (socket says 172.21.15.111:59202) 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 -- 172.21.15.111:0/269420045 learned_addr learned my addr 172.21.15.111:0/269420045 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 -- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 msgr2=0x5628f88eb5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88eb5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 -- 172.21.15.111:0/269420045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 msgr2=0x5628f88ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 -- 172.21.15.111:0/269420045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f8627620 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febd1800640 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febc000c6a0 tx=0x7febc009c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf1b7e00ef705438 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.930+0000 7febb3400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00b2020 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7febc00a4590 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb800faa0 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febc00cd270 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febd707f740 10 client.25828 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5628f88fccb0 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb800faa0 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.931+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7febc00cfce0 con 0x5628f88e2e00 2024-04-24T22:48:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 10 client.25828 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 20 client.25828 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f88fe170 0x5628f8900560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 4 client.25828 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628f8853ff0 con 0x5628f88fe170 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.932+0000 7febd0e00640 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f88fe170 0x5628f8900560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.933+0000 7febd0e00640 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f88fe170 0x5628f8900560 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5719905ec245b867 server_cookie=e3d596b427df67de in_seq=0 out_seq=0 2024-04-24T22:48:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.933+0000 7febb3400640 10 client.25828 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.933+0000 7febb3400640 1 -- 172.21.15.111:0/269420045 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628f8853ff0 con 0x5628f88fe170 2024-04-24T22:48:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.933+0000 7febb3400640 10 client.25828 handle_command_reply: tid=0 2024-04-24T22:48:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 client.25828 shutdown 2024-04-24T22:48:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 msgr2=0x5628f88eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febc000c6a0 tx=0x7febc009c2b0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 shutdown_connections 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f88fe170 0x5628f8900560 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f88ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 --2- 172.21.15.111:0/269420045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 >> 172.21.15.111:0/269420045 conn(0x5628f88df310 msgr2=0x5628f88e5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febb3400640 0 client.25828 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 shutdown_connections 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 1 -- 172.21.15.111:0/269420045 wait complete. 2024-04-24T22:48:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.934+0000 7febd707f740 20 client.25828 trim_cache size 0 max 16384 2024-04-24T22:48:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.950+0000 7febd707f740 1 Processor -- start 2024-04-24T22:48:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.950+0000 7febd707f740 1 -- start start 2024-04-24T22:48:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f8905e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1b1fa0 con 0x5628f88edfe0 2024-04-24T22:48:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1af870 con 0x5628f88e3720 2024-04-24T22:48:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd707f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1ad6e0 con 0x5628f88e2e00 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:59212/0 (socket says 172.21.15.111:59212) 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 -- 172.21.15.111:0/2474803180 learned_addr learned my addr 172.21.15.111:0/2474803180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd2200640 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f8905e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd0e00640 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 -- 172.21.15.111:0/2474803180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 msgr2=0x5628f88de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 -- 172.21.15.111:0/2474803180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 msgr2=0x5628f8905e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f8905e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.951+0000 7febd1800640 1 -- 172.21.15.111:0/2474803180 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f88e2bc0 con 0x5628f88e2e00 2024-04-24T22:48:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febd1800640 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7febc0098580 tx=0x7febc00cf970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ac4ff75ebcd1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febbe000640 1 -- 172.21.15.111:0/2474803180 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00b2070 con 0x5628f88e2e00 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febbe000640 1 -- 172.21.15.111:0/2474803180 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc007e030 con 0x5628f88e2e00 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febbe000640 1 -- 172.21.15.111:0/2474803180 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc009b220 con 0x5628f88e2e00 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febd707f740 1 -- 172.21.15.111:0/2474803180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 msgr2=0x5628f88eb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.952+0000 7febd707f740 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7febc0098580 tx=0x7febc00cf970 comp rx=0 tx=0).stop 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 -- 172.21.15.111:0/2474803180 shutdown_connections 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88edfe0 0x5628f8905e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88e3720 0x5628f88de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 --2- 172.21.15.111:0/2474803180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88eb940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 -- 172.21.15.111:0/2474803180 >> 172.21.15.111:0/2474803180 conn(0x5628f88df310 msgr2=0x5628f88ddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 -- 172.21.15.111:0/2474803180 shutdown_connections 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 -- 172.21.15.111:0/2474803180 wait complete. 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 -- start start 2024-04-24T22:48:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.953+0000 7febd707f740 1 Processor -- start 2024-04-24T22:48:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 0x5628f88f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1add30 con 0x5628f88e3720 2024-04-24T22:48:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1af1a0 con 0x5628f88edfe0 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1ae400 con 0x5628f88e2e00 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd707f740 10 client.0 _ll_register_callbacks cb 0x7febc00056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60954/0 (socket says 172.21.15.111:60954) 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 -- 172.21.15.111:0/3467581148 learned_addr learned my addr 172.21.15.111:0/3467581148 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd1800640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd2200640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 0x5628f88f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 -- 172.21.15.111:0/3467581148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 msgr2=0x5628f88f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 -- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 msgr2=0x5628f88f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 0x5628f88f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd0e00640 1 -- 172.21.15.111:0/3467581148 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f88f3540 con 0x5628f88e3720 2024-04-24T22:48:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd1800640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.954+0000 7febd2200640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 0x5628f88f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febd0e00640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7febb8011eb0 tx=0x7febb8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243a76c863c97e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febb3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb801e040 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc007e030 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7febb8011040 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febb8011260 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febd707f740 10 client.15759 fetch_fsmap learned FSMap version 24 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.955+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5628f8964f30 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc007e030 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7febb8026170 con 0x5628f88e3720 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:48:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 20 client.15759 populate_metadata read hostname 'smithi111' 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f89654f0 0x5628f8967910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 4 client.15759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628f884e340 con 0x5628f89654f0 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.956+0000 7febd1800640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f89654f0 0x5628f8967910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.957+0000 7febd1800640 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f89654f0 0x5628f8967910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c56230224fa814 server_cookie=5c13054dd937d797 in_seq=0 out_seq=0 2024-04-24T22:48:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.957+0000 7febb3400640 10 client.15759 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.971+0000 7febb3400640 1 -- 172.21.15.111:0/3467581148 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628f884e340 con 0x5628f89654f0 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.971+0000 7febb3400640 10 client.15759 handle_command_reply: tid=0 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.971+0000 7febd707f740 1 client.15759 shutdown 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 msgr2=0x5628f88f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7febb8011eb0 tx=0x7febb8005a90 comp rx=0 tx=0).stop 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 shutdown_connections 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5628f89654f0 0x5628f8967910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5628f88edfe0 0x5628f88f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5628f88e3720 0x5628f88f64f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 --2- 172.21.15.111:0/3467581148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628f88e2e00 0x5628f88f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 >> 172.21.15.111:0/3467581148 conn(0x5628f88df310 msgr2=0x5628f88ddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febb3400640 0 client.15759 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 shutdown_connections 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 1 -- 172.21.15.111:0/3467581148 wait complete. 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.972+0000 7febd707f740 20 client.15759 trim_cache size 0 max 16384 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:48:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 msgr2=0x7febb40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 0x7febb40a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febc4004af0 tx=0x7febc4005e30 comp rx=0 tx=0).stop 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 msgr2=0x7febcc19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9c0 tx=0x7febc000ee80 comp rx=0 tx=0).stop 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 shutdown_connections 2024-04-24T22:48:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febcc109950 0x7febcc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc104300 0x7febcc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 --2- 172.21.15.111:0/3764991489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febcc103940 0x7febcc19a1d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 >> 172.21.15.111:0/3764991489 conn(0x7febcc0ff410 msgr2=0x7febcc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 shutdown_connections 2024-04-24T22:48:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:48:48.973+0000 7febd4200640 1 -- 172.21.15.111:0/3764991489 wait complete. 2024-04-24T22:48:50.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:50 smithi186 ceph-mon[32753]: pgmap v211: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:50.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:50 smithi111 ceph-mon[27746]: pgmap v211: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:50.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:50 smithi171 ceph-mon[32337]: pgmap v211: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:48:52.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:52 smithi186 ceph-mon[32753]: pgmap v212: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-24T22:48:52.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:52 smithi111 ceph-mon[27746]: pgmap v212: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-24T22:48:52.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:52 smithi171 ceph-mon[32337]: pgmap v212: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-24T22:48:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:54 smithi186 ceph-mon[32753]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:54.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:54 smithi111 ceph-mon[27746]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:54.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:54 smithi171 ceph-mon[32337]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-24T22:48:56.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:56 smithi111 ceph-mon[27746]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:56 smithi171 ceph-mon[32337]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:56.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:56 smithi186 ceph-mon[32753]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:58.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:48:58 smithi111 ceph-mon[27746]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:48:58 smithi171 ceph-mon[32337]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:48:58.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:48:58 smithi186 ceph-mon[32753]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:00.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:00 smithi111 ceph-mon[27746]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:00 smithi171 ceph-mon[32337]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:00.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:00 smithi186 ceph-mon[32753]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 -- 172.21.15.111:0/928969993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e410bd10 msgr2=0x7f22e410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 --2- 172.21.15.111:0/928969993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e410bd10 0x7f22e410e1b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ca0 tx=0x7f22dc02a9e0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 -- 172.21.15.111:0/928969993 shutdown_connections 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 --2- 172.21.15.111:0/928969993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e410bd10 0x7f22e410e1b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 --2- 172.21.15.111:0/928969993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e41032f0 0x7f22e410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 --2- 172.21.15.111:0/928969993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41029d0 0x7f22e4102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.116+0000 7f22e9600640 1 -- 172.21.15.111:0/928969993 >> 172.21.15.111:0/928969993 conn(0x7f22e40777c0 msgr2=0x7f22e4077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.117+0000 7f22e9600640 1 -- 172.21.15.111:0/928969993 shutdown_connections 2024-04-24T22:49:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.117+0000 7f22e9600640 1 -- 172.21.15.111:0/928969993 wait complete. 2024-04-24T22:49:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.117+0000 7f22e9600640 1 Processor -- start 2024-04-24T22:49:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 -- start start 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 0x7f22e419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e410bd10 0x7f22e419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4111b60 con 0x7f22e41029d0 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41119e0 con 0x7f22e41032f0 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e9600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4111ce0 con 0x7f22e410bd10 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35456/0 (socket says 172.21.15.111:35456) 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e2a00640 1 -- 172.21.15.111:0/722952151 learned_addr learned my addr 172.21.15.111:0/722952151 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.118+0000 7f22e3400640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e410bd10 0x7f22e419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22dbe00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 0x7f22e419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 -- 172.21.15.111:0/722952151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e410bd10 msgr2=0x7f22e419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e410bd10 0x7f22e419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 -- 172.21.15.111:0/722952151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 msgr2=0x7f22e419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 0x7f22e419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 -- 172.21.15.111:0/722952151 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e419f130 con 0x7f22e41029d0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22dbe00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 0x7f22e419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e2a00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f22d400d950 tx=0x7f22d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22db400640 1 -- 172.21.15.111:0/722952151 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4017070 con 0x7f22e41029d0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22db400640 1 -- 172.21.15.111:0/722952151 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d4009c30 con 0x7f22e41029d0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.119+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e419f420 con 0x7f22e41029d0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.120+0000 7f22db400640 1 -- 172.21.15.111:0/722952151 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d400f6b0 con 0x7f22e41029d0 2024-04-24T22:49:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.120+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41a6cb0 con 0x7f22e41029d0 2024-04-24T22:49:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 Processor -- start 2024-04-24T22:49:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22db400640 1 -- 172.21.15.111:0/722952151 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f22d40109d0 con 0x7f22e41029d0 2024-04-24T22:49:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 -- start start 2024-04-24T22:49:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c259aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c25c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f18c1b2ab0 con 0x55f18c24ee00 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f18c15b640 con 0x55f18c259fe0 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.123+0000 7f22ec391740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f18c1aea70 con 0x55f18c24f720 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35472/0 (socket says 172.21.15.111:35472) 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 -- 172.21.15.111:0/3968080651 learned_addr learned my addr 172.21.15.111:0/3968080651 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e2a00640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c259aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c25c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 -- 172.21.15.111:0/3968080651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 msgr2=0x55f18c259aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c259aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 -- 172.21.15.111:0/3968080651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 msgr2=0x55f18c25c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c25c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22e3400640 1 -- 172.21.15.111:0/3968080651 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18bec7210 con 0x55f18c24ee00 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.124+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c25c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22e3400640 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ca0 tx=0x7f22dc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02ebe3bb8df27ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22db400640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 0x7f22c80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22d3400640 1 -- 172.21.15.111:0/3968080651 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc033050 con 0x55f18c24ee00 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22db400640 1 -- 172.21.15.111:0/722952151 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f22d40cefe0 con 0x7f22e41029d0 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22d3400640 1 -- 172.21.15.111:0/3968080651 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc047070 con 0x55f18c24ee00 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22d3400640 1 -- 172.21.15.111:0/3968080651 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc02f860 con 0x55f18c24ee00 2024-04-24T22:49:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.125+0000 7f22e2a00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 0x7f22c80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- 172.21.15.111:0/3968080651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 msgr2=0x55f18c24f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ca0 tx=0x7f22dc005d80 comp rx=0 tx=0).stop 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22e2a00640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 0x7f22c80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22d40cc7e0 tx=0x7f22d4000bc0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- 172.21.15.111:0/3968080651 shutdown_connections 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c25c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c259aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 --2- 172.21.15.111:0/3968080651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c24f1e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- 172.21.15.111:0/3968080651 >> 172.21.15.111:0/3968080651 conn(0x55f18c24b310 msgr2=0x55f18c24b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- 172.21.15.111:0/3968080651 shutdown_connections 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- 172.21.15.111:0/3968080651 wait complete. 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.126+0000 7f22ec391740 1 -- start start 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 Processor -- start 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c2574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c2579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f18c25cbb0 con 0x55f18c24ee00 2024-04-24T22:49:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f18c16e990 con 0x55f18c259fe0 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f18c25cd30 con 0x55f18c24f720 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22ec391740 10 client.0 _ll_register_callbacks cb 0x55f18c170950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22e3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22e3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35486/0 (socket says 172.21.15.111:35486) 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.127+0000 7f22e3400640 1 -- 172.21.15.111:0/3709710007 learned_addr learned my addr 172.21.15.111:0/3709710007 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c2574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c2579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 -- 172.21.15.111:0/3709710007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 msgr2=0x55f18c2574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c2574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 -- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 msgr2=0x55f18c2579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c2579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 -- 172.21.15.111:0/3709710007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18bf93620 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22e3400640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f22dc02afd0 tx=0x7f22dc02e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52437bde5022483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.128+0000 7f22c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc02f840 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f22dc02f9e0 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22dc047070 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f22dc05f050 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22ec391740 10 client.15771 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f18c268a60 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc047070 con 0x55f18c24ee00 2024-04-24T22:49:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.129+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f22dc04b450 con 0x55f18c24ee00 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 10 client.15771 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 20 client.15771 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c269db0 0x55f18c26c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 4 client.15771 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.130+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18c1bfff0 con 0x55f18c269db0 2024-04-24T22:49:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.131+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c269db0 0x55f18c26c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.133+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c269db0 0x55f18c26c1a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b3da52c4b85b8a server_cookie=fd2c0baeb8bb8f0 in_seq=0 out_seq=0 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.133+0000 7f22c7e00640 10 client.15771 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.134+0000 7f22c7e00640 1 -- 172.21.15.111:0/3709710007 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f18c1bfff0 con 0x55f18c269db0 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.134+0000 7f22c7e00640 10 client.15771 handle_command_reply: tid=0 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 client.15771 shutdown 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 msgr2=0x55f18c256f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f22dc02afd0 tx=0x7f22dc02e720 comp rx=0 tx=0).stop 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 shutdown_connections 2024-04-24T22:49:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c269db0 0x55f18c26c1a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c259fe0 0x55f18c2579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c2574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22c7e00640 0 client.15771 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 --2- 172.21.15.111:0/3709710007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24ee00 0x55f18c256f60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 >> 172.21.15.111:0/3709710007 conn(0x55f18c24b310 msgr2=0x55f18c251040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 shutdown_connections 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 1 -- 172.21.15.111:0/3709710007 wait complete. 2024-04-24T22:49:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.135+0000 7f22ec391740 20 client.15771 trim_cache size 0 max 16384 2024-04-24T22:49:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.151+0000 7f22ec391740 1 Processor -- start 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.151+0000 7f22ec391740 1 -- start start 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c265a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c259fe0 0x55f18c257800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41b1d90 con 0x55f18c259fe0 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41af660 con 0x55f18c24ee00 2024-04-24T22:49:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22ec391740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41ad4d0 con 0x55f18c24f720 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:50180/0 (socket says 172.21.15.111:50180) 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.152+0000 7f22dbe00640 1 -- 172.21.15.111:0/3232807167 learned_addr learned my addr 172.21.15.111:0/3232807167 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22e3400640 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c265a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 -- 172.21.15.111:0/3232807167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 msgr2=0x55f18c265a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c265a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 -- 172.21.15.111:0/3232807167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c259fe0 msgr2=0x55f18c257800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c259fe0 0x55f18c257800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 -- 172.21.15.111:0/3232807167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c24fe90 con 0x55f18c24f720 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22e3400640 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c265a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22dbe00640 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009cb0 tx=0x7f22cc013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0baf0129f0c898 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.153+0000 7f22d2a00640 1 -- 172.21.15.111:0/3232807167 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc01e670 con 0x55f18c24f720 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22d2a00640 1 -- 172.21.15.111:0/3232807167 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22cc01e810 con 0x55f18c24f720 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22d2a00640 1 -- 172.21.15.111:0/3232807167 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc01d420 con 0x55f18c24f720 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 -- 172.21.15.111:0/3232807167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 msgr2=0x55f18c255700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009cb0 tx=0x7f22cc013830 comp rx=0 tx=0).stop 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 -- 172.21.15.111:0/3232807167 shutdown_connections 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c259fe0 0x55f18c257800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c24f720 0x55f18c255700 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 --2- 172.21.15.111:0/3232807167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c265a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.154+0000 7f22ec391740 1 -- 172.21.15.111:0/3232807167 >> 172.21.15.111:0/3232807167 conn(0x55f18c24b310 msgr2=0x55f18c251040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.155+0000 7f22ec391740 1 -- 172.21.15.111:0/3232807167 shutdown_connections 2024-04-24T22:49:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.155+0000 7f22ec391740 1 -- 172.21.15.111:0/3232807167 wait complete. 2024-04-24T22:49:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.155+0000 7f22ec391740 1 -- start start 2024-04-24T22:49:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.155+0000 7f22ec391740 1 Processor -- start 2024-04-24T22:49:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c2622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c259fe0 0x55f18c25db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41adb20 con 0x55f18c24f720 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41aef90 con 0x55f18c24ee00 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41ae1f0 con 0x55f18c259fe0 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22ec391740 10 client.0 _ll_register_callbacks cb 0x7f22dc02eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22e3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c2622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35494/0 (socket says 172.21.15.111:35494) 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22e3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c2622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:47860/0 (socket says 172.21.15.111:47860) 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 -- 172.21.15.111:0/1511375475 learned_addr learned my addr 172.21.15.111:0/1511375475 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 -- 172.21.15.111:0/1511375475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c259fe0 msgr2=0x55f18c25db70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c259fe0 0x55f18c25db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 -- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 msgr2=0x55f18c2622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c2622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.156+0000 7f22dbe00640 1 -- 172.21.15.111:0/1511375475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c25e320 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22dbe00640 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22cc01e640 tx=0x7f22cc01e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e402def761ab03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc004980 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f22cc004cb0 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22cc01e810 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f22cc037050 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22ec391740 10 client.15774 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f18c2cfee0 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.157+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22cc01e810 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f22cc0278c0 con 0x55f18c24f720 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 10 client.15774 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 20 client.15774 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c2d0600 0x55f18c2d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 4 client.15774 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f18c1ba340 con 0x55f18c2d0600 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.158+0000 7f22e3400640 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c2d0600 0x55f18c2d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.159+0000 7f22c7e00640 10 client.15774 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.159+0000 7f22e3400640 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c2d0600 0x55f18c2d2a20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b531aedaccd9635b server_cookie=2b1d837690ce5347 in_seq=0 out_seq=0 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.176+0000 7f22c7e00640 1 -- 172.21.15.111:0/1511375475 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f18c1ba340 con 0x55f18c2d0600 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.176+0000 7f22c7e00640 10 client.15774 handle_command_reply: tid=0 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 client.15774 shutdown 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 msgr2=0x55f18c25d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22cc01e640 tx=0x7f22cc01e670 comp rx=0 tx=0).stop 2024-04-24T22:49:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 shutdown_connections 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f18c2d0600 0x55f18c2d2a20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f18c259fe0 0x55f18c25db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f18c24f720 0x55f18c25d630 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 --2- 172.21.15.111:0/1511375475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f18c24ee00 0x55f18c2622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22c7e00640 0 client.15774 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 >> 172.21.15.111:0/1511375475 conn(0x55f18c24b310 msgr2=0x55f18c249bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 shutdown_connections 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 1 -- 172.21.15.111:0/1511375475 wait complete. 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.177+0000 7f22ec391740 20 client.15774 trim_cache size 0 max 16384 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:49:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:49:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:49:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:49:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 msgr2=0x7f22c80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 0x7f22c80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22d40cc7e0 tx=0x7f22d4000bc0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 msgr2=0x7f22e419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f22d400d950 tx=0x7f22d400de10 comp rx=0 tx=0).stop 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 shutdown_connections 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f22c80a2e20 0x7f22c80a52e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e410bd10 0x7f22e419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22e41032f0 0x7f22e419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 --2- 172.21.15.111:0/722952151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e41029d0 0x7f22e419a140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.178+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 >> 172.21.15.111:0/722952151 conn(0x7f22e40777c0 msgr2=0x7f22e40fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.179+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 shutdown_connections 2024-04-24T22:49:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:02.179+0000 7f22e9600640 1 -- 172.21.15.111:0/722952151 wait complete. 2024-04-24T22:49:02.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:02 smithi111 ceph-mon[27746]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:02 smithi171 ceph-mon[32337]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:02.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:02 smithi186 ceph-mon[32753]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:04 smithi111 ceph-mon[27746]: pgmap v218: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-04-24T22:49:04.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:04 smithi171 ceph-mon[32337]: pgmap v218: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-04-24T22:49:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:04 smithi186 ceph-mon[32753]: pgmap v218: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-04-24T22:49:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:06 smithi111 ceph-mon[27746]: pgmap v219: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-04-24T22:49:06.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:06 smithi186 ceph-mon[32753]: pgmap v219: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-04-24T22:49:06.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:06 smithi171 ceph-mon[32337]: pgmap v219: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-04-24T22:49:07.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:07 smithi171 ceph-mon[32337]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T22:49:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:07 smithi186 ceph-mon[32753]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T22:49:07.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:07 smithi111 ceph-mon[27746]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T22:49:08.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:08 smithi171 ceph-mon[32337]: pgmap v221: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:49:08.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:08 smithi186 ceph-mon[32753]: pgmap v221: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:49:08.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:08 smithi111 ceph-mon[27746]: pgmap v221: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:49:10.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:10 smithi171 ceph-mon[32337]: pgmap v222: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:10.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:10 smithi186 ceph-mon[32753]: pgmap v222: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:10.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:10 smithi111 ceph-mon[27746]: pgmap v222: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:12.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:12 smithi171 ceph-mon[32337]: pgmap v223: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:12.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:12 smithi171 ceph-mon[32337]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T22:49:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:12 smithi186 ceph-mon[32753]: pgmap v223: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:12 smithi186 ceph-mon[32753]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T22:49:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:12 smithi111 ceph-mon[27746]: pgmap v223: 97 pgs: 97 active+clean; 4.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-04-24T22:49:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:12 smithi111 ceph-mon[27746]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T22:49:14.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:14 smithi171 ceph-mon[32337]: pgmap v225: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:49:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:14 smithi186 ceph-mon[32753]: pgmap v225: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:49:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:14 smithi111 ceph-mon[27746]: pgmap v225: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:49:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 -- 172.21.15.111:0/373391302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 msgr2=0x7fd52810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 --2- 172.21.15.111:0/373391302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52810daf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd51402a490 comp rx=0 tx=0).stop 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 -- 172.21.15.111:0/373391302 shutdown_connections 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 --2- 172.21.15.111:0/373391302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd52810e030 0x7fd528110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 --2- 172.21.15.111:0/373391302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52810daf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 --2- 172.21.15.111:0/373391302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd528073a40 0x7fd528073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 -- 172.21.15.111:0/373391302 >> 172.21.15.111:0/373391302 conn(0x7fd528078bd0 msgr2=0x7fd528078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 -- 172.21.15.111:0/373391302 shutdown_connections 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.320+0000 7fd530400640 1 -- 172.21.15.111:0/373391302 wait complete. 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 Processor -- start 2024-04-24T22:49:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 -- start start 2024-04-24T22:49:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd52810e030 0x7fd52819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd528113db0 con 0x7fd528073a40 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd528113c30 con 0x7fd528074360 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.322+0000 7fd530400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd528113f30 con 0x7fd52810e030 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52106/0 (socket says 172.21.15.111:52106) 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 -- 172.21.15.111:0/2716983876 learned_addr learned my addr 172.21.15.111:0/2716983876 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52d000640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52e400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd52810e030 0x7fd52819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 -- 172.21.15.111:0/2716983876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd52810e030 msgr2=0x7fd52819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd52810e030 0x7fd52819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 -- 172.21.15.111:0/2716983876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 msgr2=0x7fd52819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 -- 172.21.15.111:0/2716983876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52819f1d0 con 0x7fd528073a40 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.323+0000 7fd52da00640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9f0 tx=0x7fd51c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.324+0000 7fd526000640 1 -- 172.21.15.111:0/2716983876 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00cd80 con 0x7fd528073a40 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.324+0000 7fd526000640 1 -- 172.21.15.111:0/2716983876 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c005500 con 0x7fd528073a40 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.324+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52819f4c0 con 0x7fd528073a40 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.324+0000 7fd526000640 1 -- 172.21.15.111:0/2716983876 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c010690 con 0x7fd528073a40 2024-04-24T22:49:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.324+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5281a6d00 con 0x7fd528073a40 2024-04-24T22:49:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.327+0000 7fd5331a3740 1 Processor -- start 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.327+0000 7fd5331a3740 1 -- start start 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.327+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33debaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd526000640 1 -- 172.21.15.111:0/2716983876 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd51c0046c0 con 0x7fd528073a40 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd5331a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fa33d44ab0 con 0x55fa33de0e00 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd5331a3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fa33ced640 con 0x55fa33debfe0 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd5331a3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa33d40a70 con 0x55fa33de1720 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:46980/0 (socket says 172.21.15.111:46980) 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 -- 172.21.15.111:0/2869032004 learned_addr learned my addr 172.21.15.111:0/2869032004 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52da00640 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33debaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 -- 172.21.15.111:0/2869032004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 msgr2=0x55fa33debaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33debaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 -- 172.21.15.111:0/2869032004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 msgr2=0x55fa33de11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.328+0000 7fd52d000640 1 -- 172.21.15.111:0/2869032004 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa33a59210 con 0x55fa33debfe0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd52d000640 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd514005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a531d9cec2e057d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd51aa00640 1 -- 172.21.15.111:0/2869032004 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514035040 con 0x55fa33debfe0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd51aa00640 1 -- 172.21.15.111:0/2869032004 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd514004400 con 0x55fa33debfe0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd51aa00640 1 -- 172.21.15.111:0/2869032004 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140035f0 con 0x55fa33debfe0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd526000640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 0x7fd5100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.329+0000 7fd526000640 1 -- 172.21.15.111:0/2716983876 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fd51c0cf7d0 con 0x7fd528073a40 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd52e400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 0x7fd5100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- 172.21.15.111:0/2869032004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 msgr2=0x55fa33dee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd514005de0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- 172.21.15.111:0/2869032004 shutdown_connections 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33dee3d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33debaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2869032004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- 172.21.15.111:0/2869032004 >> 172.21.15.111:0/2869032004 conn(0x55fa33ddd310 msgr2=0x55fa33ddd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- 172.21.15.111:0/2869032004 shutdown_connections 2024-04-24T22:49:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- 172.21.15.111:0/2869032004 wait complete. 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd5331a3740 1 -- start start 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.330+0000 7fd52e400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 0x7fd5100a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd520005fd0 tx=0x7fd520009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 Processor -- start 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33de94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33de99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fa33deebb0 con 0x55fa33de0e00 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fa33d00990 con 0x55fa33debfe0 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa33deed30 con 0x55fa33de1720 2024-04-24T22:49:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd5331a3740 10 client.0 _ll_register_callbacks cb 0x55fa33d02950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd52da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd52da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52130/0 (socket says 172.21.15.111:52130) 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd52da00640 1 -- 172.21.15.111:0/4204214696 learned_addr learned my addr 172.21.15.111:0/4204214696 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.331+0000 7fd52e400640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33de99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52d000640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33de94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 -- 172.21.15.111:0/4204214696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 msgr2=0x55fa33de94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33de94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 -- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 msgr2=0x55fa33de99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33de99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 -- 172.21.15.111:0/4204214696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa33b25620 con 0x55fa33de0e00 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd52da00640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c6a0 tx=0x7fd51c09bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670a25ca7a473df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd50f400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0a46d0 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fd51c09b140 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.332+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd514004400 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd51c09bcf0 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 10 client.15786 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fa33dfaac0 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd514004400 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd51c0a4bb0 con 0x55fa33de0e00 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 20 client.15786 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33dfbf30 0x55fa33dfe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 4 client.15786 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.333+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa33d51ff0 con 0x55fa33dfbf30 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.334+0000 7fd52d000640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33dfbf30 0x55fa33dfe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.334+0000 7fd52d000640 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33dfbf30 0x55fa33dfe320 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28901bc8f7e09002 server_cookie=cb593b49712520d2 in_seq=0 out_seq=0 2024-04-24T22:49:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.334+0000 7fd50f400640 10 client.15786 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd50f400640 1 -- 172.21.15.111:0/4204214696 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa33d51ff0 con 0x55fa33dfbf30 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd50f400640 10 client.15786 handle_command_reply: tid=0 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 client.15786 shutdown 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 msgr2=0x55fa33de8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c6a0 tx=0x7fd51c09bb80 comp rx=0 tx=0).stop 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 shutdown_connections 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33dfbf30 0x55fa33dfe320 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33debfe0 0x55fa33de99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33de1720 0x55fa33de94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 --2- 172.21.15.111:0/4204214696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de0e00 0x55fa33de8f60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd50f400640 0 client.15786 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.335+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 >> 172.21.15.111:0/4204214696 conn(0x55fa33ddd310 msgr2=0x55fa33de3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.336+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 shutdown_connections 2024-04-24T22:49:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.336+0000 7fd5331a3740 1 -- 172.21.15.111:0/4204214696 wait complete. 2024-04-24T22:49:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.336+0000 7fd5331a3740 20 client.15786 trim_cache size 0 max 16384 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 Processor -- start 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 -- start start 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33de9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b1db0 con 0x55fa33de1720 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281af680 con 0x55fa33de0e00 2024-04-24T22:49:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd5331a3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ad4f0 con 0x55fa33debfe0 2024-04-24T22:49:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd52d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd52d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52138/0 (socket says 172.21.15.111:52138) 2024-04-24T22:49:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd52d000640 1 -- 172.21.15.111:0/180505209 learned_addr learned my addr 172.21.15.111:0/180505209 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.352+0000 7fd52da00640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52e400640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33de9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 -- 172.21.15.111:0/180505209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 msgr2=0x55fa33de9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33de9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 -- 172.21.15.111:0/180505209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 msgr2=0x55fa33ddc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 -- 172.21.15.111:0/180505209 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa33dfe900 con 0x55fa33de0e00 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52d000640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd52da00640 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df7a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0986c0 tx=0x7fd51c0d5240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb9e7fa857498c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd51a000640 1 -- 172.21.15.111:0/180505209 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0a00a0 con 0x55fa33de0e00 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd51a000640 1 -- 172.21.15.111:0/180505209 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c0a4960 con 0x55fa33de0e00 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.353+0000 7fd51a000640 1 -- 172.21.15.111:0/180505209 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0a4730 con 0x55fa33de0e00 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 -- 172.21.15.111:0/180505209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 msgr2=0x55fa33df7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df7a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0986c0 tx=0x7fd51c0d5240 comp rx=0 tx=0).stop 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 -- 172.21.15.111:0/180505209 shutdown_connections 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33de9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33ddc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 --2- 172.21.15.111:0/180505209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df7a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 -- 172.21.15.111:0/180505209 >> 172.21.15.111:0/180505209 conn(0x55fa33ddd310 msgr2=0x55fa33dea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 -- 172.21.15.111:0/180505209 shutdown_connections 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.354+0000 7fd5331a3740 1 -- 172.21.15.111:0/180505209 wait complete. 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 -- start start 2024-04-24T22:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 Processor -- start 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33df05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33e613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281adb40 con 0x55fa33de1720 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281aefb0 con 0x55fa33de0e00 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ae210 con 0x55fa33debfe0 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.355+0000 7fd5331a3740 10 client.0 _ll_register_callbacks cb 0x7fd51c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:47004/0 (socket says 172.21.15.111:47004) 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 -- 172.21.15.111:0/2299299757 learned_addr learned my addr 172.21.15.111:0/2299299757 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52d000640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33df05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52e400640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33e613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 -- 172.21.15.111:0/2299299757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 msgr2=0x55fa33e613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33e613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 -- 172.21.15.111:0/2299299757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 msgr2=0x55fa33df05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33df05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 -- 172.21.15.111:0/2299299757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa33e61920 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd52da00640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd51c09b9f0 tx=0x7fd51c09bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57b0b8cec811de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.356+0000 7fd50f400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c09b650 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fd51c097a40 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51c0a4960 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd51c0a00a0 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 10 client.25826 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fa33e61920 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c0a4960 con 0x55fa33de0e00 2024-04-24T22:49:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd51c004f40 con 0x55fa33de0e00 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 10 client.25826 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 20 client.25826 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33e62ef0 0x55fa33e652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 4 client.25826 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.357+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa33d4c340 con 0x55fa33e62ef0 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.358+0000 7fd52d000640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33e62ef0 0x55fa33e652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.358+0000 7fd52d000640 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33e62ef0 0x55fa33e652e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fac3b12ea16f77 server_cookie=ad121e8acce0b3e6 in_seq=0 out_seq=0 2024-04-24T22:49:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.358+0000 7fd50f400640 10 client.25826 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.373+0000 7fd50f400640 1 -- 172.21.15.111:0/2299299757 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa33d4c340 con 0x55fa33e62ef0 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.373+0000 7fd50f400640 10 client.25826 handle_command_reply: tid=0 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.373+0000 7fd5331a3740 1 client.25826 shutdown 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 msgr2=0x55fa33df00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd51c09b9f0 tx=0x7fd51c09bbf0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 shutdown_connections 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fa33e62ef0 0x55fa33e652e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa33debfe0 0x55fa33e613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa33de1720 0x55fa33df05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 --2- 172.21.15.111:0/2299299757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa33de0e00 0x55fa33df00b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd50f400640 0 client.25826 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 >> 172.21.15.111:0/2299299757 conn(0x55fa33ddd310 msgr2=0x55fa33ddc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 shutdown_connections 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 1 -- 172.21.15.111:0/2299299757 wait complete. 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.374+0000 7fd5331a3740 20 client.25826 trim_cache size 0 max 16384 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:49:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 msgr2=0x7fd5100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 0x7fd5100a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd520005fd0 tx=0x7fd520009500 comp rx=0 tx=0).stop 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 msgr2=0x7fd52819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9f0 tx=0x7fd51c00edf0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 shutdown_connections 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fd5100a2e20 0x7fd5100a52e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd52810e030 0x7fd52819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd528074360 0x7fd52819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 --2- 172.21.15.111:0/2716983876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd528073a40 0x7fd52819a190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 >> 172.21.15.111:0/2716983876 conn(0x7fd528078bd0 msgr2=0x7fd5280ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 shutdown_connections 2024-04-24T22:49:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:15.375+0000 7fd530400640 1 -- 172.21.15.111:0/2716983876 wait complete. 2024-04-24T22:49:16.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:16 smithi171 ceph-mon[32337]: pgmap v226: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:49:16.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:16 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:16 smithi186 ceph-mon[32753]: pgmap v226: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:49:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:16 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:16.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:16 smithi111 ceph-mon[27746]: pgmap v226: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:49:16.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:16 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:17.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:17 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:17.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:17 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:17.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:17 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:49:17.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:17 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:17.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:17 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:17.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:17 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:49:17.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:17 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:17.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:17 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:17.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:17 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:49:18.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:18 smithi171 ceph-mon[32337]: pgmap v227: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:49:18.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:18 smithi186 ceph-mon[32753]: pgmap v227: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:49:18.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:18 smithi111 ceph-mon[27746]: pgmap v227: 97 pgs: 97 active+clean; 4.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:49:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:20 smithi171 ceph-mon[32337]: pgmap v228: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:20.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:20 smithi186 ceph-mon[32753]: pgmap v228: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:20.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:20 smithi111 ceph-mon[27746]: pgmap v228: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:22 smithi171 ceph-mon[32337]: pgmap v229: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:22.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:22 smithi186 ceph-mon[32753]: pgmap v229: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:22.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:22 smithi111 ceph-mon[27746]: pgmap v229: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-04-24T22:49:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:24 smithi171 ceph-mon[32337]: pgmap v230: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:49:24.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:24 smithi186 ceph-mon[32753]: pgmap v230: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:49:24.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:24 smithi111 ceph-mon[27746]: pgmap v230: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:49:26.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:26 smithi171 ceph-mon[32337]: pgmap v231: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:26 smithi186 ceph-mon[32753]: pgmap v231: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:26 smithi111 ceph-mon[27746]: pgmap v231: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 -- 172.21.15.111:0/3218494904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc1030c0 msgr2=0x7f36cc10b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 --2- 172.21.15.111:0/3218494904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc1030c0 0x7f36cc10b5a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009ca0 tx=0x7f36bc02a950 comp rx=0 tx=0).stop 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 -- 172.21.15.111:0/3218494904 shutdown_connections 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 --2- 172.21.15.111:0/3218494904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc10bb70 0x7f36cc10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 --2- 172.21.15.111:0/3218494904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc1030c0 0x7f36cc10b5a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 --2- 172.21.15.111:0/3218494904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1027a0 0x7f36cc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.518+0000 7f36d2200640 1 -- 172.21.15.111:0/3218494904 >> 172.21.15.111:0/3218494904 conn(0x7f36cc0fbf80 msgr2=0x7f36cc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.519+0000 7f36d2200640 1 -- 172.21.15.111:0/3218494904 shutdown_connections 2024-04-24T22:49:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.519+0000 7f36d2200640 1 -- 172.21.15.111:0/3218494904 wait complete. 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 Processor -- start 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 -- start start 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 0x7f36cc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1030c0 0x7f36cc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc111a10 con 0x7f36cc10bb70 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc111890 con 0x7f36cc1030c0 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d2200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc111b90 con 0x7f36cc1027a0 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48406/0 (socket says 172.21.15.111:48406) 2024-04-24T22:49:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36d1800640 1 -- 172.21.15.111:0/1831220876 learned_addr learned my addr 172.21.15.111:0/1831220876 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.520+0000 7f36cbe00640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1030c0 0x7f36cc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 0x7f36cc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 -- 172.21.15.111:0/1831220876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 msgr2=0x7f36cc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 0x7f36cc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 -- 172.21.15.111:0/1831220876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1030c0 msgr2=0x7f36cc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1030c0 0x7f36cc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 -- 172.21.15.111:0/1831220876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc19cfc0 con 0x7f36cc10bb70 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 0x7f36cc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d1800640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009fd0 tx=0x7f36c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36c9600640 1 -- 172.21.15.111:0/1831220876 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4011830 con 0x7f36cc10bb70 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36c9600640 1 -- 172.21.15.111:0/1831220876 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4016070 con 0x7f36cc10bb70 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.521+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc19d2b0 con 0x7f36cc10bb70 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.522+0000 7f36c9600640 1 -- 172.21.15.111:0/1831220876 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c400f2e0 con 0x7f36cc10bb70 2024-04-24T22:49:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.522+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1a4b40 con 0x7f36cc10bb70 2024-04-24T22:49:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.525+0000 7f36c9600640 1 -- 172.21.15.111:0/1831220876 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f36c40119d0 con 0x7f36cc10bb70 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.525+0000 7f36d6590740 1 Processor -- start 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.525+0000 7f36d6590740 1 -- start start 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 0x55733d070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55733cfc4ab0 con 0x55733d0653c0 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55733cf6d640 con 0x55733d0705a0 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d6590740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55733cfc0a70 con 0x55733d065ce0 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48412/0 (socket says 172.21.15.111:48412) 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36cbe00640 1 -- 172.21.15.111:0/1314055104 learned_addr learned my addr 172.21.15.111:0/1314055104 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 0x55733d070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36cbe00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.526+0000 7f36d0e00640 1 -- 172.21.15.111:0/1314055104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 msgr2=0x55733d072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36d0e00640 1 -- 172.21.15.111:0/1314055104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 msgr2=0x55733d0657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36d0e00640 1 -- 172.21.15.111:0/1314055104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733ccd9210 con 0x55733d065ce0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36cbe00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36d0e00640 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 0x55733d070060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36b400a940 tx=0x7f36b400ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad885646ed9204b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36c2000640 1 -- 172.21.15.111:0/1314055104 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4015070 con 0x55733d065ce0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36c9600640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 0x7f36b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36c2000640 1 -- 172.21.15.111:0/1314055104 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b400cc00 con 0x55733d065ce0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36c2000640 1 -- 172.21.15.111:0/1314055104 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b40185c0 con 0x55733d065ce0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36c9600640 1 -- 172.21.15.111:0/1831220876 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f36c40ce9a0 con 0x7f36cc10bb70 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.527+0000 7f36cbe00640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 0x7f36b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 -- 172.21.15.111:0/1314055104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 msgr2=0x55733d070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 0x55733d070060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36b400a940 tx=0x7f36b400ae00 comp rx=0 tx=0).stop 2024-04-24T22:49:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 -- 172.21.15.111:0/1314055104 shutdown_connections 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d065ce0 0x55733d070060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 --2- 172.21.15.111:0/1314055104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0653c0 0x55733d0657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 -- 172.21.15.111:0/1314055104 >> 172.21.15.111:0/1314055104 conn(0x55733d0618d0 msgr2=0x55733d061cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 -- 172.21.15.111:0/1314055104 shutdown_connections 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36cbe00640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 0x7f36b00a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36bc030040 tx=0x7f36bc005fa0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.528+0000 7f36d6590740 1 -- 172.21.15.111:0/1314055104 wait complete. 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 -- start start 2024-04-24T22:49:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 Processor -- start 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d06df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55733d0731d0 con 0x55733d0705a0 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55733cf80990 con 0x55733d065ce0 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55733d073350 con 0x55733d0653c0 2024-04-24T22:49:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.529+0000 7f36d6590740 10 client.0 _ll_register_callbacks cb 0x55733cf82950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:45594/0 (socket says 172.21.15.111:45594) 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 -- 172.21.15.111:0/3664285633 learned_addr learned my addr 172.21.15.111:0/3664285633 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d1800640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d06df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 -- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 msgr2=0x55733d06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 -- 172.21.15.111:0/3664285633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 msgr2=0x55733d06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d06df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 -- 172.21.15.111:0/3664285633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55733cda5620 con 0x55733d0653c0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d1800640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d06df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36d0e00640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f36b4008010 tx=0x7f36b40119e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc066735a0d12c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.530+0000 7f36baa00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4029060 con 0x55733d0653c0 2024-04-24T22:49:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b400cc00 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f36b401a450 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36b4015040 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36d6590740 10 client.25873 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55733d07f040 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b400cc00 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f36b402c530 con 0x55733d0653c0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.531+0000 7f36d6590740 10 client.25873 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36d6590740 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36d6590740 20 client.25873 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d080390 0x55733d082780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36d6590740 4 client.25873 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55733cfd1ff0 con 0x55733d080390 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.532+0000 7f36cbe00640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d080390 0x55733d082780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.533+0000 7f36cbe00640 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d080390 0x55733d082780 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2c85d00b5a2a77 server_cookie=9be2ebbbae3bc3ca in_seq=0 out_seq=0 2024-04-24T22:49:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.533+0000 7f36baa00640 10 client.25873 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36baa00640 1 -- 172.21.15.111:0/3664285633 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55733cfd1ff0 con 0x55733d080390 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36baa00640 10 client.25873 handle_command_reply: tid=0 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 client.25873 shutdown 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 msgr2=0x55733d06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f36b4008010 tx=0x7f36b40119e0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 shutdown_connections 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d080390 0x55733d082780 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d06df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36baa00640 0 client.25873 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d06da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 --2- 172.21.15.111:0/3664285633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d06d4c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 >> 172.21.15.111:0/3664285633 conn(0x55733d0618d0 msgr2=0x55733d05c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 shutdown_connections 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 1 -- 172.21.15.111:0/3664285633 wait complete. 2024-04-24T22:49:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.534+0000 7f36d6590740 20 client.25873 trim_cache size 0 max 16384 2024-04-24T22:49:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.550+0000 7f36d6590740 1 Processor -- start 2024-04-24T22:49:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.550+0000 7f36d6590740 1 -- start start 2024-04-24T22:49:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d07c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 0x55733d05c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1ada70 con 0x55733d065ce0 2024-04-24T22:49:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1ab340 con 0x55733d0705a0 2024-04-24T22:49:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d6590740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1a91b0 con 0x55733d0653c0 2024-04-24T22:49:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 0x55733d05c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:56308/0 (socket says 172.21.15.111:56308) 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 0x55733d05c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48426/0 (socket says 172.21.15.111:48426) 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d1800640 1 -- 172.21.15.111:0/2251333873 learned_addr learned my addr 172.21.15.111:0/2251333873 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.551+0000 7f36d0e00640 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d07c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 -- 172.21.15.111:0/2251333873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 msgr2=0x55733d087e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 -- 172.21.15.111:0/2251333873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 msgr2=0x55733d05c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 0x55733d05c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 -- 172.21.15.111:0/2251333873 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733d065b10 con 0x55733d0653c0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d1800640 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36d0e00640 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d07c090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36b402cb40 tx=0x7f36b400c5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4f544ac51444b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36c1600640 1 -- 172.21.15.111:0/2251333873 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4029050 con 0x55733d0653c0 2024-04-24T22:49:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36c1600640 1 -- 172.21.15.111:0/2251333873 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b402cbc0 con 0x55733d0653c0 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.552+0000 7f36c1600640 1 -- 172.21.15.111:0/2251333873 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b401a450 con 0x55733d0653c0 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- 172.21.15.111:0/2251333873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 msgr2=0x55733d07c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d07c090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36b402cb40 tx=0x7f36b400c5a0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- 172.21.15.111:0/2251333873 shutdown_connections 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d0705a0 0x55733d087e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d065ce0 0x55733d05c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 --2- 172.21.15.111:0/2251333873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d07c090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- 172.21.15.111:0/2251333873 >> 172.21.15.111:0/2251333873 conn(0x55733d0618d0 msgr2=0x55733d05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- 172.21.15.111:0/2251333873 shutdown_connections 2024-04-24T22:49:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- 172.21.15.111:0/2251333873 wait complete. 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.553+0000 7f36d6590740 1 -- start start 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 Processor -- start 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d074680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1a9800 con 0x55733d0705a0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1aac70 con 0x55733d065ce0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1a9ed0 con 0x55733d0653c0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d6590740 10 client.0 _ll_register_callbacks cb 0x7f36bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.554+0000 7f36d1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48430/0 (socket says 172.21.15.111:48430) 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d1800640 1 -- 172.21.15.111:0/306396929 learned_addr learned my addr 172.21.15.111:0/306396929 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36cbe00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 -- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 msgr2=0x55733d074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d074680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 -- 172.21.15.111:0/306396929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 msgr2=0x55733d074bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d074bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 -- 172.21.15.111:0/306396929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55733d0e5860 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36d0e00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d074140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1720c29a730da989 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.555+0000 7f36baa00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4029040 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f36b402c3b0 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b402cbc0 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36b402c650 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36d6590740 10 client.25879 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55733d0e6b30 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.556+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b402cbc0 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f36b401a4e0 con 0x55733d0653c0 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 10 client.25879 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 20 client.25879 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d0e7130 0x55733d0e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 4 client.25879 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:49:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55733cfcc340 con 0x55733d0e7130 2024-04-24T22:49:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.557+0000 7f36cbe00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d0e7130 0x55733d0e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.558+0000 7f36cbe00640 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d0e7130 0x55733d0e9520 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e54a971f76a1fc server_cookie=256ce11108a56b9a in_seq=0 out_seq=0 2024-04-24T22:49:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.558+0000 7f36baa00640 10 client.25879 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.572+0000 7f36baa00640 1 -- 172.21.15.111:0/306396929 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55733cfcc340 con 0x55733d0e7130 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.572+0000 7f36baa00640 10 client.25879 handle_command_reply: tid=0 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.572+0000 7f36d6590740 1 client.25879 shutdown 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.572+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 msgr2=0x55733d074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.572+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d074140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4011e30 comp rx=0 tx=0).stop 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 shutdown_connections 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55733d0e7130 0x55733d0e9520 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55733d0705a0 0x55733d074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55733d065ce0 0x55733d074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36baa00640 0 client.25879 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 --2- 172.21.15.111:0/306396929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55733d0653c0 0x55733d074140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 >> 172.21.15.111:0/306396929 conn(0x55733d0618d0 msgr2=0x55733d05c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 shutdown_connections 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 1 -- 172.21.15.111:0/306396929 wait complete. 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.573+0000 7f36d6590740 20 client.25879 trim_cache size 0 max 16384 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:49:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 msgr2=0x7f36b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 0x7f36b00a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36bc030040 tx=0x7f36bc005fa0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 msgr2=0x7f36cc19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009fd0 tx=0x7f36c400b8e0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 shutdown_connections 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36cc10bb70 0x7f36cc19c890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36cc1030c0 0x7f36cc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f36b00a2cf0 0x7f36b00a51b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 --2- 172.21.15.111:0/1831220876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36cc1027a0 0x7f36cc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 >> 172.21.15.111:0/1831220876 conn(0x7f36cc0fbf80 msgr2=0x7f36cc0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 shutdown_connections 2024-04-24T22:49:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:28.574+0000 7f36d2200640 1 -- 172.21.15.111:0/1831220876 wait complete. 2024-04-24T22:49:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:28 smithi171 ceph-mon[32337]: pgmap v232: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:28.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:28 smithi186 ceph-mon[32753]: pgmap v232: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:28.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:28 smithi111 ceph-mon[27746]: pgmap v232: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-24T22:49:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:30 smithi171 ceph-mon[32337]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:49:30.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:30 smithi186 ceph-mon[32753]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:49:30.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:30 smithi111 ceph-mon[27746]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-24T22:49:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:32 smithi171 ceph-mon[32337]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:49:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:32 smithi186 ceph-mon[32753]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:49:32.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:32 smithi111 ceph-mon[27746]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:49:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:34 smithi171 ceph-mon[32337]: pgmap v235: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-24T22:49:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:34 smithi186 ceph-mon[32753]: pgmap v235: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-24T22:49:34.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:34 smithi111 ceph-mon[27746]: pgmap v235: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-24T22:49:36.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:36 smithi171 ceph-mon[32337]: pgmap v236: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:36.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:36 smithi186 ceph-mon[32753]: pgmap v236: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:36.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:36 smithi111 ceph-mon[27746]: pgmap v236: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:36.832 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:313/600: open fd = 3 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:314/600: open fd = 3 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:315/600: open fd = 3 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.833 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:316/600: open fd = 3 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:317/600: open fd = 3 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:318/600: open fd = 3 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.834 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:319/600: open fd = 3 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:320/600: open fd = 3 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:321/600: open fd = 3 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.835 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:322/600: open fd = 3 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:323/600: open fd = 3 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.836 INFO:tasks.workunit.client.0.smithi111.stdout:324/600: open fd = 3 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:325/600: open fd = 3 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:326/600: open fd = 3 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:327/600: open fd = 3 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.837 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:328/600: open fd = 3 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:329/600: open fd = 3 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:330/600: open fd = 3 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.838 INFO:tasks.workunit.client.0.smithi111.stdout:331/600: open fd = 3 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:332/600: open fd = 3 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:333/600: open fd = 3 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:334/600: open fd = 3 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.839 INFO:tasks.workunit.client.0.smithi111.stdout:335/600: open fd = 3 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:336/600: open fd = 3 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:337/600: open fd = 3 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:338/600: open fd = 3 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.840 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:339/600: open fd = 3 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:340/600: open fd = 3 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:341/600: open fd = 3 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.841 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:342/600: open fd = 3 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:343/600: open fd = 3 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:344/600: open fd = 3 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.842 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:345/600: open fd = 3 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:346/600: open fd = 3 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:347/600: open fd = 3 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:348/600: open fd = 3 2024-04-24T22:49:36.843 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:349/600: open fd = 3 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:350/600: open fd = 3 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:351/600: open fd = 3 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.844 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:352/600: open fd = 3 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:353/600: open fd = 3 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:354/600: open fd = 3 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:355/600: open fd = 3 2024-04-24T22:49:36.845 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:356/600: open fd = 3 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:357/600: open fd = 3 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:358/600: open fd = 3 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.846 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:359/600: open fd = 3 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:360/600: open fd = 3 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:361/600: open fd = 3 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.847 INFO:tasks.workunit.client.0.smithi111.stdout:362/600: open fd = 3 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:363/600: open fd = 3 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:364/600: open fd = 3 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:365/600: open fd = 3 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.848 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:366/600: open fd = 3 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:367/600: open fd = 3 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:368/600: open fd = 3 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:369/600: open fd = 3 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.849 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:370/600: open fd = 3 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:371/600: open fd = 3 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:372/600: open fd = 3 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.850 INFO:tasks.workunit.client.0.smithi111.stdout:373/600: open fd = 3 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:374/600: open fd = 3 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:49:36.851 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:49:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:38 smithi171 ceph-mon[32337]: pgmap v237: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:38 smithi186 ceph-mon[32753]: pgmap v237: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:38.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:38 smithi111 ceph-mon[27746]: pgmap v237: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-24T22:49:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:40 smithi171 ceph-mon[32337]: pgmap v238: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:49:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:40 smithi186 ceph-mon[32753]: pgmap v238: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:49:40.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:40 smithi111 ceph-mon[27746]: pgmap v238: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 -- 172.21.15.111:0/578281181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c103930 msgr2=0x7f120c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 --2- 172.21.15.111:0/578281181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c103930 0x7f120c103db0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009930 tx=0x7f11f402a440 comp rx=0 tx=0).stop 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 -- 172.21.15.111:0/578281181 shutdown_connections 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 --2- 172.21.15.111:0/578281181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f120c1042f0 0x7f120c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 --2- 172.21.15.111:0/578281181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c103930 0x7f120c103db0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 --2- 172.21.15.111:0/578281181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f120c109940 0x7f120c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 -- 172.21.15.111:0/578281181 >> 172.21.15.111:0/578281181 conn(0x7f120c0ff440 msgr2=0x7f120c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 -- 172.21.15.111:0/578281181 shutdown_connections 2024-04-24T22:49:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.717+0000 7f1210c00640 1 -- 172.21.15.111:0/578281181 wait complete. 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 Processor -- start 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 -- start start 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f120c103930 0x7f120c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f120c109940 0x7f120c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113dc0 con 0x7f120c1042f0 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113c40 con 0x7f120c103930 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.718+0000 7f1210c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113f40 con 0x7f120c109940 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50720/0 (socket says 172.21.15.111:50720) 2024-04-24T22:49:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 -- 172.21.15.111:0/2541079022 learned_addr learned my addr 172.21.15.111:0/2541079022 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 -- 172.21.15.111:0/2541079022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f120c109940 msgr2=0x7f120c19eb80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f120c109940 0x7f120c19eb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 -- 172.21.15.111:0/2541079022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f120c103930 msgr2=0x7f120c19a2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f120c103930 0x7f120c19a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.719+0000 7f1209400640 1 -- 172.21.15.111:0/2541079022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19f2b0 con 0x7f120c1042f0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1209400640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11f402a960 tx=0x7f11f4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1202a00640 1 -- 172.21.15.111:0/2541079022 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f403f070 con 0x7f120c1042f0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1202a00640 1 -- 172.21.15.111:0/2541079022 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f4031c90 con 0x7f120c1042f0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1202a00640 1 -- 172.21.15.111:0/2541079022 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4035030 con 0x7f120c1042f0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19f540 con 0x7f120c1042f0 2024-04-24T22:49:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.720+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a6e30 con 0x7f120c1042f0 2024-04-24T22:49:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.724+0000 7f12139f2740 1 Processor -- start 2024-04-24T22:49:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.724+0000 7f12139f2740 1 -- start start 2024-04-24T22:49:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.724+0000 7f1202a00640 1 -- 172.21.15.111:0/2541079022 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f11f4042070 con 0x7f120c1042f0 2024-04-24T22:49:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ec31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ecdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55caf9e26ab0 con 0x55caf9ecdfe0 2024-04-24T22:49:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55caf9dcf640 con 0x55caf9ec3720 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f12139f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55caf9e22a70 con 0x55caf9ec2e00 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f1209400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f1209400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50726/0 (socket says 172.21.15.111:50726) 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f1209400640 1 -- 172.21.15.111:0/3121957735 learned_addr learned my addr 172.21.15.111:0/3121957735 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.725+0000 7f120a800640 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ec31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 -- 172.21.15.111:0/3121957735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 msgr2=0x55caf9ec31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ec31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 -- 172.21.15.111:0/3121957735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 msgr2=0x55caf9ecdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ecdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 -- 172.21.15.111:0/3121957735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf9b3b210 con 0x55caf9ecdfe0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1209400640 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ed640 tx=0x7f11f40f83b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35df0b54b56e815 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1202a00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 0x7f11f00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f1202a00640 1 -- 172.21.15.111:0/2541079022 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f11f40efff0 con 0x7f120c1042f0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f120a800640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 0x7f11f00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.726+0000 7f11fb000640 1 -- 172.21.15.111:0/3121957735 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40b8770 con 0x55caf9ecdfe0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f11fb000640 1 -- 172.21.15.111:0/3121957735 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f40fd030 con 0x55caf9ecdfe0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f11fb000640 1 -- 172.21.15.111:0/3121957735 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40f3c90 con 0x55caf9ecdfe0 2024-04-24T22:49:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 -- 172.21.15.111:0/3121957735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 msgr2=0x55caf9ed03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ed640 tx=0x7f11f40f83b0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f120a800640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 0x7f11f00a52e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009370 tx=0x7f11fc0095d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 -- 172.21.15.111:0/3121957735 shutdown_connections 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed03d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ecdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 --2- 172.21.15.111:0/3121957735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ec31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 -- 172.21.15.111:0/3121957735 >> 172.21.15.111:0/3121957735 conn(0x55caf9ebf310 msgr2=0x55caf9ebf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 -- 172.21.15.111:0/3121957735 shutdown_connections 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.727+0000 7f12139f2740 1 -- 172.21.15.111:0/3121957735 wait complete. 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.728+0000 7f12139f2740 1 -- start start 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.728+0000 7f12139f2740 1 Processor -- start 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.728+0000 7f12139f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.728+0000 7f12139f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ecb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.728+0000 7f12139f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ecdfe0 0x55caf9ecba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f12139f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55caf9ed09d0 con 0x55caf9ec2e00 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f12139f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55caf9de2990 con 0x55caf9ecdfe0 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f12139f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55caf9ed0b50 con 0x55caf9ec3720 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f12139f2740 10 client.0 _ll_register_callbacks cb 0x55caf9de4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50732/0 (socket says 172.21.15.111:50732) 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 -- 172.21.15.111:0/2601740594 learned_addr learned my addr 172.21.15.111:0/2601740594 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209400640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ecb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 -- 172.21.15.111:0/2601740594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 msgr2=0x55caf9ecb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ecb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 -- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ecdfe0 msgr2=0x55caf9ecba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ecdfe0 0x55caf9ecba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.729+0000 7f1209e00640 1 -- 172.21.15.111:0/2601740594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf9c07620 con 0x55caf9ec2e00 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f1209e00640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1204009dd0 tx=0x7f120400b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb26a485df5ff47d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f11ef400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204015020 con 0x55caf9ec2e00 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f1204011420 con 0x55caf9ec2e00 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f40fd030 con 0x55caf9ec2e00 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1204020050 con 0x55caf9ec2e00 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f12139f2740 10 client.15822 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55caf9edc910 con 0x55caf9ec2e00 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.730+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f40fd030 con 0x55caf9ec2e00 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f12040198c0 con 0x55caf9ec2e00 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 10 client.15822 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 20 client.15822 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9edde20 0x55caf9ee0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 4 client.15822 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf9e33ff0 con 0x55caf9edde20 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.731+0000 7f1209400640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9edde20 0x55caf9ee0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.732+0000 7f1209400640 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9edde20 0x55caf9ee0290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0fc91bee79c732 server_cookie=e0a83d3c07fb1fec in_seq=0 out_seq=0 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.732+0000 7f11ef400640 10 client.15822 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f11ef400640 1 -- 172.21.15.111:0/2601740594 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caf9e33ff0 con 0x55caf9edde20 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f11ef400640 10 client.15822 handle_command_reply: tid=0 2024-04-24T22:49:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 client.15822 shutdown 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 msgr2=0x55caf9ecafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1204009dd0 tx=0x7f120400b4b0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 shutdown_connections 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9edde20 0x55caf9ee0290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ecdfe0 0x55caf9ecba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ecb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 --2- 172.21.15.111:0/2601740594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ec2e00 0x55caf9ecafd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f11ef400640 0 client.15822 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 >> 172.21.15.111:0/2601740594 conn(0x55caf9ebf310 msgr2=0x55caf9ec5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 shutdown_connections 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 1 -- 172.21.15.111:0/2601740594 wait complete. 2024-04-24T22:49:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.733+0000 7f12139f2740 20 client.15822 trim_cache size 0 max 16384 2024-04-24T22:49:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.749+0000 7f12139f2740 1 Processor -- start 2024-04-24T22:49:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.749+0000 7f12139f2740 1 -- start start 2024-04-24T22:49:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec2e00 0x55caf9ecb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ebe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1b1f70 con 0x55caf9ecdfe0 2024-04-24T22:49:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1af840 con 0x55caf9ec2e00 2024-04-24T22:49:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f12139f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1ad6d0 con 0x55caf9ec3720 2024-04-24T22:49:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f120a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f120a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50746/0 (socket says 172.21.15.111:50746) 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f120a800640 1 -- 172.21.15.111:0/1139135217 learned_addr learned my addr 172.21.15.111:0/1139135217 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f1209e00640 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec2e00 0x55caf9ecb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.750+0000 7f1209400640 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ebe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 -- 172.21.15.111:0/1139135217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 msgr2=0x55caf9ebe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ebe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 -- 172.21.15.111:0/1139135217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec2e00 msgr2=0x55caf9ecb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec2e00 0x55caf9ecb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 -- 172.21.15.111:0/1139135217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf9ebd890 con 0x55caf9ecdfe0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f120a800640 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f11fc004960 tx=0x7f11fc013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aea42dba7ee44db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f11fa600640 1 -- 172.21.15.111:0/1139135217 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01e620 con 0x55caf9ecdfe0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f11fa600640 1 -- 172.21.15.111:0/1139135217 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc01eca0 con 0x55caf9ecdfe0 2024-04-24T22:49:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.751+0000 7f11fa600640 1 -- 172.21.15.111:0/1139135217 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01cb90 con 0x55caf9ecdfe0 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- 172.21.15.111:0/1139135217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 msgr2=0x55caf9ee5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f11fc004960 tx=0x7f11fc013070 comp rx=0 tx=0).stop 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- 172.21.15.111:0/1139135217 shutdown_connections 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ee5ba0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec3720 0x55caf9ebe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 --2- 172.21.15.111:0/1139135217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec2e00 0x55caf9ecb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- 172.21.15.111:0/1139135217 >> 172.21.15.111:0/1139135217 conn(0x55caf9ebf310 msgr2=0x55caf9ebdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- 172.21.15.111:0/1139135217 shutdown_connections 2024-04-24T22:49:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- 172.21.15.111:0/1139135217 wait complete. 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.752+0000 7f12139f2740 1 -- start start 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 Processor -- start 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ed61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1add20 con 0x55caf9ecdfe0 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1af170 con 0x55caf9ec3720 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1ae3d0 con 0x55caf9ec2e00 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f12139f2740 10 client.0 _ll_register_callbacks cb 0x7f1204006e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f120a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.753+0000 7f1209e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:59150/0 (socket says 172.21.15.111:59150) 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 -- 172.21.15.111:0/1808449465 learned_addr learned my addr 172.21.15.111:0/1808449465 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209400640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ed61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 -- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 msgr2=0x55caf9ed61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ed61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 -- 172.21.15.111:0/1808449465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 msgr2=0x55caf9ed2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 -- 172.21.15.111:0/1808449465 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf9ed3330 con 0x55caf9ec2e00 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209400640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ed61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f1209e00640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1204019cb0 tx=0x7f120400ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4872a7dc1d0b6a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.754+0000 7f11ef400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204028020 con 0x55caf9ec2e00 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f1204004450 con 0x55caf9ec2e00 2024-04-24T22:49:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc01eca0 con 0x55caf9ec2e00 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1204033050 con 0x55caf9ec2e00 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f12139f2740 10 client.25891 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.755+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55caf9f44db0 con 0x55caf9ec2e00 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc01eca0 con 0x55caf9ec2e00 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1204003a20 con 0x55caf9ec2e00 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 10 client.25891 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 20 client.25891 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9f455b0 0x55caf9f479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 4 client.25891 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:49:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55caf9e2e340 con 0x55caf9f455b0 2024-04-24T22:49:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.756+0000 7f1209400640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9f455b0 0x55caf9f479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.757+0000 7f1209400640 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9f455b0 0x55caf9f479d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ea5bd6a0c30d6 server_cookie=3b56fc5caa494a0d in_seq=0 out_seq=0 2024-04-24T22:49:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.757+0000 7f11ef400640 10 client.25891 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f11ef400640 1 -- 172.21.15.111:0/1808449465 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55caf9e2e340 con 0x55caf9f455b0 2024-04-24T22:49:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f11ef400640 10 client.25891 handle_command_reply: tid=0 2024-04-24T22:49:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 client.25891 shutdown 2024-04-24T22:49:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 msgr2=0x55caf9ed5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1204019cb0 tx=0x7f120400ba80 comp rx=0 tx=0).stop 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 shutdown_connections 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55caf9f455b0 0x55caf9f479d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55caf9ecdfe0 0x55caf9ed2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55caf9ec3720 0x55caf9ed61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 --2- 172.21.15.111:0/1808449465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf9ec2e00 0x55caf9ed5ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f11ef400640 0 client.25891 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 >> 172.21.15.111:0/1808449465 conn(0x55caf9ebf310 msgr2=0x55caf9ebe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 shutdown_connections 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.773+0000 7f12139f2740 1 -- 172.21.15.111:0/1808449465 wait complete. 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.774+0000 7f12139f2740 20 client.25891 trim_cache size 0 max 16384 2024-04-24T22:49:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:49:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:49:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:49:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:49:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.774+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 msgr2=0x7f11f00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.774+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 0x7f11f00a52e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009370 tx=0x7f11fc0095d0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.774+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 msgr2=0x7f120c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.774+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11f402a960 tx=0x7f11f4003b90 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 shutdown_connections 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f11f00a2e20 0x7f11f00a52e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f120c109940 0x7f120c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1042f0 0x7f120c19a800 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 --2- 172.21.15.111:0/2541079022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f120c103930 0x7f120c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 >> 172.21.15.111:0/2541079022 conn(0x7f120c0ff440 msgr2=0x7f120c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 shutdown_connections 2024-04-24T22:49:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:41.775+0000 7f1210c00640 1 -- 172.21.15.111:0/2541079022 wait complete. 2024-04-24T22:49:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:42 smithi171 ceph-mon[32337]: pgmap v239: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:42 smithi186 ceph-mon[32753]: pgmap v239: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:42.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:42 smithi111 ceph-mon[27746]: pgmap v239: 97 pgs: 97 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:44.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:44 smithi171 ceph-mon[32337]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-04-24T22:49:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:44 smithi186 ceph-mon[32753]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-04-24T22:49:44.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:44 smithi111 ceph-mon[27746]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-04-24T22:49:46.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:46 smithi171 ceph-mon[32337]: pgmap v241: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:46 smithi186 ceph-mon[32753]: pgmap v241: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:46.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:46 smithi111 ceph-mon[27746]: pgmap v241: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:48 smithi171 ceph-mon[32337]: pgmap v242: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:48 smithi186 ceph-mon[32753]: pgmap v242: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:48.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:48 smithi111 ceph-mon[27746]: pgmap v242: 97 pgs: 97 active+clean; 2.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:49:50.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:50 smithi171 ceph-mon[32337]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:50.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:50 smithi186 ceph-mon[32753]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:50.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:50 smithi111 ceph-mon[27746]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:51.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:51 smithi186 ceph-mon[32753]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:51.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:51 smithi111 ceph-mon[27746]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:51.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:51 smithi171 ceph-mon[32337]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:49:54.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:53 smithi111 ceph-mon[27746]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:54.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:53 smithi171 ceph-mon[32337]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:54.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:53 smithi186 ceph-mon[32753]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:49:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 -- 172.21.15.111:0/3042579277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 msgr2=0x7f3bac110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 --2- 172.21.15.111:0/3042579277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac110810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009cd0 tx=0x7f3ba402a8f0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 -- 172.21.15.111:0/3042579277 shutdown_connections 2024-04-24T22:49:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 --2- 172.21.15.111:0/3042579277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac110810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 --2- 172.21.15.111:0/3042579277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac103940 0x7f3bac103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 --2- 172.21.15.111:0/3042579277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac109950 0x7f3bac109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 -- 172.21.15.111:0/3042579277 >> 172.21.15.111:0/3042579277 conn(0x7f3bac0ff470 msgr2=0x7f3bac101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 -- 172.21.15.111:0/3042579277 shutdown_connections 2024-04-24T22:49:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.918+0000 7f3bb3000640 1 -- 172.21.15.111:0/3042579277 wait complete. 2024-04-24T22:49:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.919+0000 7f3bb3000640 1 Processor -- start 2024-04-24T22:49:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.919+0000 7f3bb3000640 1 -- start start 2024-04-24T22:49:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac109950 0x7f3bac19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac113d60 con 0x7f3bac103940 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac113be0 con 0x7f3bac109950 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb3000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac113ee0 con 0x7f3bac104300 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60134/0 (socket says 172.21.15.111:60134) 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 -- 172.21.15.111:0/693726861 learned_addr learned my addr 172.21.15.111:0/693726861 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bb1000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac109950 0x7f3bac19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3bab400640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 -- 172.21.15.111:0/693726861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 msgr2=0x7f3bac19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 -- 172.21.15.111:0/693726861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac109950 msgr2=0x7f3bac19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.920+0000 7f3babe00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac109950 0x7f3bac19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3babe00640 1 -- 172.21.15.111:0/693726861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac19f1f0 con 0x7f3bac103940 2024-04-24T22:49:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3babe00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d950 tx=0x7f3b9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3ba8c00640 1 -- 172.21.15.111:0/693726861 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c017030 con 0x7f3bac103940 2024-04-24T22:49:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3ba8c00640 1 -- 172.21.15.111:0/693726861 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c004020 con 0x7f3bac103940 2024-04-24T22:49:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3ba8c00640 1 -- 172.21.15.111:0/693726861 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0047a0 con 0x7f3bac103940 2024-04-24T22:49:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac19f4e0 con 0x7f3bac103940 2024-04-24T22:49:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.921+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a6d70 con 0x7f3bac103940 2024-04-24T22:49:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.924+0000 7f3bb5e30740 1 Processor -- start 2024-04-24T22:49:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3ba8c00640 1 -- 172.21.15.111:0/693726861 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3b9c0041d0 con 0x7f3bac103940 2024-04-24T22:49:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 -- start start 2024-04-24T22:49:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d766aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d6bfab0 con 0x55dc5d75be00 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d668640 con 0x55dc5d75c720 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d6bba70 con 0x55dc5d766fe0 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60144/0 (socket says 172.21.15.111:60144) 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3bb1000640 1 -- 172.21.15.111:0/3201682125 learned_addr learned my addr 172.21.15.111:0/3201682125 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.925+0000 7f3babe00640 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d766aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bab400640 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 -- 172.21.15.111:0/3201682125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 msgr2=0x55dc5d7693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 -- 172.21.15.111:0/3201682125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 msgr2=0x55dc5d766aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d766aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 -- 172.21.15.111:0/3201682125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5d3d4210 con 0x55dc5d75be00 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4035000 tx=0x7f3ba4005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83cdbdce61d75a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3ba8c00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 0x7f3b900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.926+0000 7f3ba1600640 1 -- 172.21.15.111:0/3201682125 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4030760 con 0x55dc5d75be00 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3ba1600640 1 -- 172.21.15.111:0/3201682125 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4036030 con 0x55dc5d75be00 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3ba8c00640 1 -- 172.21.15.111:0/693726861 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f3b9c0cecd0 con 0x7f3bac103940 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3ba1600640 1 -- 172.21.15.111:0/3201682125 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004460 con 0x55dc5d75be00 2024-04-24T22:49:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3babe00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 0x7f3b900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3201682125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 msgr2=0x55dc5d75c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3babe00640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 0x7f3b900a52e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55dc5d75d3a0 tx=0x7f3b9c0d2780 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.927+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4035000 tx=0x7f3ba4005fa0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3201682125 shutdown_connections 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d766aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3201682125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d75c1e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3201682125 >> 172.21.15.111:0/3201682125 conn(0x55dc5d758310 msgr2=0x55dc5d7586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3201682125 shutdown_connections 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3201682125 wait complete. 2024-04-24T22:49:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 -- start start 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.928+0000 7f3bb5e30740 1 Processor -- start 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d7644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d769bb0 con 0x55dc5d75be00 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d67b990 con 0x55dc5d75c720 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d769d30 con 0x55dc5d766fe0 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb5e30740 10 client.0 _ll_register_callbacks cb 0x55dc5d67d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60152/0 (socket says 172.21.15.111:60152) 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb1000640 1 -- 172.21.15.111:0/2299204284 learned_addr learned my addr 172.21.15.111:0/2299204284 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bb1000640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.929+0000 7f3bab400640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d7644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 -- 172.21.15.111:0/2299204284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 msgr2=0x55dc5d7649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 -- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 msgr2=0x55dc5d7644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d7644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 -- 172.21.15.111:0/2299204284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5d4a0620 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb1000640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009620 tx=0x7f3ba4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec24b1e90c1cef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3b96000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4059060 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f3ba40043e0 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.930+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4036030 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ba4030d70 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 10 client.15840 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc5d769bb0 con 0x55dc5d75be00 2024-04-24T22:49:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4036030 con 0x55dc5d75be00 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3ba4043070 con 0x55dc5d75be00 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 10 client.15840 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 20 client.15840 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d776be0 0x55dc5d778fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 4 client.15840 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.931+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc5d6ccff0 con 0x55dc5d776be0 2024-04-24T22:49:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.932+0000 7f3bab400640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d776be0 0x55dc5d778fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.935+0000 7f3bab400640 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d776be0 0x55dc5d778fd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3d3e9b00c0dc34 server_cookie=d253776c82407318 in_seq=0 out_seq=0 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.935+0000 7f3b96000640 10 client.15840 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.936+0000 7f3b96000640 1 -- 172.21.15.111:0/2299204284 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc5d6ccff0 con 0x55dc5d776be0 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.936+0000 7f3b96000640 10 client.15840 handle_command_reply: tid=0 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.936+0000 7f3bb5e30740 1 client.15840 shutdown 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.936+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 msgr2=0x55dc5d763f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.936+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009620 tx=0x7f3ba4003f50 comp rx=0 tx=0).stop 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 shutdown_connections 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d776be0 0x55dc5d778fd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d766fe0 0x55dc5d7649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d7644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3b96000640 0 client.15840 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/2299204284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d763f60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 >> 172.21.15.111:0/2299204284 conn(0x55dc5d758310 msgr2=0x55dc5d75e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 shutdown_connections 2024-04-24T22:49:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 1 -- 172.21.15.111:0/2299204284 wait complete. 2024-04-24T22:49:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.937+0000 7f3bb5e30740 20 client.15840 trim_cache size 0 max 16384 2024-04-24T22:49:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.953+0000 7f3bb5e30740 1 Processor -- start 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 -- start start 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 0x55dc5d766730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d764800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b1ee0 con 0x55dc5d75be00 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1af7b0 con 0x55dc5d75c720 2024-04-24T22:49:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1ad620 con 0x55dc5d758310 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60168/0 (socket says 172.21.15.111:60168) 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 0x55dc5d766730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bab400640 1 -- 172.21.15.111:0/1678471920 learned_addr learned my addr 172.21.15.111:0/1678471920 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.954+0000 7f3bb1000640 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d764800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 -- 172.21.15.111:0/1678471920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 msgr2=0x55dc5d764800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d764800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 -- 172.21.15.111:0/1678471920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 msgr2=0x55dc5d772a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 -- 172.21.15.111:0/1678471920 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5d756890 con 0x55dc5d758310 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bab400640 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3bb1000640 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 0x55dc5d766730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4003a10 tx=0x7f3ba402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae67c485839b5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.955+0000 7f3ba0c00640 1 -- 172.21.15.111:0/1678471920 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4059030 con 0x55dc5d758310 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3ba0c00640 1 -- 172.21.15.111:0/1678471920 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4036040 con 0x55dc5d758310 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3ba0c00640 1 -- 172.21.15.111:0/1678471920 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4003fc0 con 0x55dc5d758310 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 -- 172.21.15.111:0/1678471920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 msgr2=0x55dc5d766730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 0x55dc5d766730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4003a10 tx=0x7f3ba402a8f0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 -- 172.21.15.111:0/1678471920 shutdown_connections 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d75c720 0x55dc5d764800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d772a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/1678471920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d758310 0x55dc5d766730 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 -- 172.21.15.111:0/1678471920 >> 172.21.15.111:0/1678471920 conn(0x55dc5d75e520 msgr2=0x55dc5d766020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 -- 172.21.15.111:0/1678471920 shutdown_connections 2024-04-24T22:49:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.956+0000 7f3bb5e30740 1 -- 172.21.15.111:0/1678471920 wait complete. 2024-04-24T22:49:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 -- start start 2024-04-24T22:49:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 Processor -- start 2024-04-24T22:49:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 0x55dc5d76bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d75c720 0x55dc5d76c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1adc70 con 0x55dc5d75be00 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1af0e0 con 0x55dc5d758310 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.957+0000 7f3bb5e30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1ae340 con 0x55dc5d75c720 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bb5e30740 10 client.0 _ll_register_callbacks cb 0x7f3b9c0d2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 0x55dc5d76bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bb1000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 0x55dc5d76bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:33964/0 (socket says 172.21.15.111:33964) 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bb1000640 1 -- 172.21.15.111:0/3360011180 learned_addr learned my addr 172.21.15.111:0/3360011180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60184/0 (socket says 172.21.15.111:60184) 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d75c720 0x55dc5d76c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 -- 172.21.15.111:0/3360011180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d75c720 msgr2=0x55dc5d76c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d75c720 0x55dc5d76c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 -- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 msgr2=0x55dc5d76bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 0x55dc5d76bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.958+0000 7f3bab400640 1 -- 172.21.15.111:0/3360011180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5d7dceb0 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3bab400640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3b98004700 tx=0x7f3b9800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a02578a57799a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3b96000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801d580 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f3b9801d720 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4036040 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3b9801d920 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3bb5e30740 10 client.15846 fetch_fsmap learned FSMap version 24 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc5d7de100 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.959+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4036040 con 0x55dc5d75be00 2024-04-24T22:49:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3b98011560 con 0x55dc5d75be00 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 10 client.15846 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 20 client.15846 populate_metadata read hostname 'smithi111' 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d766fe0 0x55dc5d7e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 4 client.15846 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc5d6c7340 con 0x55dc5d766fe0 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d766fe0 0x55dc5d7e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.961+0000 7f3b96000640 10 client.15846 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.960+0000 7f3bb1000640 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d766fe0 0x55dc5d7e0870 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30ce29e99b4064a server_cookie=aca4857217dbade in_seq=0 out_seq=0 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3b96000640 1 -- 172.21.15.111:0/3360011180 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc5d6c7340 con 0x55dc5d766fe0 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3b96000640 10 client.15846 handle_command_reply: tid=0 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 client.15846 shutdown 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 msgr2=0x55dc5d76c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3b98004700 tx=0x7f3b9800bfc0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 shutdown_connections 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55dc5d766fe0 0x55dc5d7e0870 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc5d75c720 0x55dc5d76c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dc5d75be00 0x55dc5d76c490 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 --2- 172.21.15.111:0/3360011180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc5d758310 0x55dc5d76bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3b96000640 0 client.15846 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 >> 172.21.15.111:0/3360011180 conn(0x55dc5d75e520 msgr2=0x55dc5d771a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 shutdown_connections 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.977+0000 7f3bb5e30740 1 -- 172.21.15.111:0/3360011180 wait complete. 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.978+0000 7f3bb5e30740 20 client.15846 trim_cache size 0 max 16384 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:49:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:49:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:49:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 msgr2=0x7f3b900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 0x7f3b900a52e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55dc5d75d3a0 tx=0x7f3b9c0d2780 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 msgr2=0x7f3bac19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d950 tx=0x7f3b9c00de10 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 shutdown_connections 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bac109950 0x7f3bac19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3b900a2e20 0x7f3b900a52e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bac104300 0x7f3bac19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 --2- 172.21.15.111:0/693726861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bac103940 0x7f3bac19a200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 >> 172.21.15.111:0/693726861 conn(0x7f3bac0ff470 msgr2=0x7f3bac0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 shutdown_connections 2024-04-24T22:49:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:49:54.979+0000 7f3bb3000640 1 -- 172.21.15.111:0/693726861 wait complete. 2024-04-24T22:49:56.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:55 smithi111 ceph-mon[27746]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:49:56.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:55 smithi171 ceph-mon[32337]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:49:56.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:55 smithi186 ceph-mon[32753]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:49:58.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:57 smithi171 ceph-mon[32337]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:49:58.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:57 smithi186 ceph-mon[32753]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:49:58.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:57 smithi111 ceph-mon[27746]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:50:00.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:49:59 smithi171 ceph-mon[32337]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:50:00.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:49:59 smithi186 ceph-mon[32753]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:50:00.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:49:59 smithi111 ceph-mon[27746]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:50:01.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:00 smithi171 ceph-mon[32337]: overall HEALTH_OK 2024-04-24T22:50:01.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:00 smithi186 ceph-mon[32753]: overall HEALTH_OK 2024-04-24T22:50:01.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:00 smithi111 ceph-mon[27746]: overall HEALTH_OK 2024-04-24T22:50:02.058 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:01 smithi171 ceph-mon[32337]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:02.185 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:01 smithi111 ceph-mon[27746]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:02.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:01 smithi186 ceph-mon[32753]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:04.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:03 smithi171 ceph-mon[32337]: pgmap v250: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-04-24T22:50:04.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:03 smithi186 ceph-mon[32753]: pgmap v250: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-04-24T22:50:04.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:03 smithi111 ceph-mon[27746]: pgmap v250: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-04-24T22:50:06.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:05 smithi171 ceph-mon[32337]: pgmap v251: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 29 op/s 2024-04-24T22:50:06.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:05 smithi186 ceph-mon[32753]: pgmap v251: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 29 op/s 2024-04-24T22:50:06.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:05 smithi111 ceph-mon[27746]: pgmap v251: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 29 op/s 2024-04-24T22:50:07.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:06 smithi171 ceph-mon[32337]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T22:50:07.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:06 smithi186 ceph-mon[32753]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T22:50:07.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:06 smithi111 ceph-mon[27746]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 -- 172.21.15.111:0/4170358950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c103940 msgr2=0x7f457c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 --2- 172.21.15.111:0/4170358950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c103940 0x7f457c103dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4564009930 tx=0x7f456402a420 comp rx=0 tx=0).stop 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 -- 172.21.15.111:0/4170358950 shutdown_connections 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 --2- 172.21.15.111:0/4170358950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c104300 0x7f457c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 --2- 172.21.15.111:0/4170358950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c103940 0x7f457c103dc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 --2- 172.21.15.111:0/4170358950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c109950 0x7f457c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 -- 172.21.15.111:0/4170358950 >> 172.21.15.111:0/4170358950 conn(0x7f457c0ff470 msgr2=0x7f457c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 -- 172.21.15.111:0/4170358950 shutdown_connections 2024-04-24T22:50:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.122+0000 7f4583000640 1 -- 172.21.15.111:0/4170358950 wait complete. 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.123+0000 7f4583000640 1 Processor -- start 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.123+0000 7f4583000640 1 -- start start 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c104300 0x7f457c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c109950 0x7f457c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f457c113db0 con 0x7f457c104300 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f457c113c30 con 0x7f457c109950 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4583000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f457c113f30 con 0x7f457c103940 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4573e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c104300 0x7f457c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:38626/0 (socket says 172.21.15.111:38626) 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 -- 172.21.15.111:0/1754224627 learned_addr learned my addr 172.21.15.111:0/1754224627 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f4581000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c109950 0x7f457c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 -- 172.21.15.111:0/1754224627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c109950 msgr2=0x7f457c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c109950 0x7f457c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 -- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c104300 msgr2=0x7f457c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c104300 0x7f457c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.124+0000 7f457be00640 1 -- 172.21.15.111:0/1754224627 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c19f210 con 0x7f457c103940 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f457be00640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9f0 tx=0x7f456c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f4579600640 1 -- 172.21.15.111:0/1754224627 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c00ccc0 con 0x7f457c103940 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f4579600640 1 -- 172.21.15.111:0/1754224627 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c00ce60 con 0x7f457c103940 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c19f500 con 0x7f457c103940 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f4579600640 1 -- 172.21.15.111:0/1754224627 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c010690 con 0x7f457c103940 2024-04-24T22:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.125+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c1a6d90 con 0x7f457c103940 2024-04-24T22:50:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.128+0000 7f4585e42740 1 Processor -- start 2024-04-24T22:50:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.128+0000 7f4579600640 1 -- 172.21.15.111:0/1754224627 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f456c0046c0 con 0x7f457c103940 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.128+0000 7f4585e42740 1 -- start start 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dbeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564179d17ab0 con 0x564179dbefe0 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564179cc0640 con 0x564179db4720 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4585e42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564179d13a70 con 0x564179db3e00 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f457be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dbeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:38630/0 (socket says 172.21.15.111:38630) 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4573e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 -- 172.21.15.111:0/3681607098 learned_addr learned my addr 172.21.15.111:0/3681607098 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 -- 172.21.15.111:0/3681607098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 msgr2=0x564179dbeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dbeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 -- 172.21.15.111:0/3681607098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 msgr2=0x564179dc13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f4581000640 1 -- 172.21.15.111:0/3681607098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564179a2c210 con 0x564179db3e00 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.129+0000 7f457be00640 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4581000640 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4574011440 tx=0x7f4574011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c71d9a5db81537 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4579600640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 0x7f45600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4571600640 1 -- 172.21.15.111:0/3681607098 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574003c30 con 0x564179db3e00 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4571600640 1 -- 172.21.15.111:0/3681607098 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574003dd0 con 0x564179db3e00 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4571600640 1 -- 172.21.15.111:0/3681607098 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400faf0 con 0x564179db3e00 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f4579600640 1 -- 172.21.15.111:0/1754224627 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8557+0+0 (secure 0 0 0) 0x7f456c0cf7d0 con 0x7f457c103940 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.130+0000 7f457be00640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 0x7f45600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- 172.21.15.111:0/3681607098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 msgr2=0x564179db41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4574011440 tx=0x7f4574011900 comp rx=0 tx=0).stop 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- 172.21.15.111:0/3681607098 shutdown_connections 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f457be00640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 0x7f45600a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ccfb0 tx=0x7f456c0d5550 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 --2- 172.21.15.111:0/3681607098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179db41e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- 172.21.15.111:0/3681607098 >> 172.21.15.111:0/3681607098 conn(0x564179db0310 msgr2=0x564179db06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- 172.21.15.111:0/3681607098 shutdown_connections 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- 172.21.15.111:0/3681607098 wait complete. 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.131+0000 7f4585e42740 1 -- start start 2024-04-24T22:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 Processor -- start 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179dbc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564179dc1c10 con 0x564179dbefe0 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564179cd3990 con 0x564179db3e00 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564179dc1d90 con 0x564179db4720 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.132+0000 7f4585e42740 10 client.0 _ll_register_callbacks cb 0x564179cd5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4581000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179dbc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4581000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:58836/0 (socket says 172.21.15.111:58836) 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4581000640 1 -- 172.21.15.111:0/361105234 learned_addr learned my addr 172.21.15.111:0/361105234 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4581000640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dbca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 -- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 msgr2=0x564179dbbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 -- 172.21.15.111:0/361105234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 msgr2=0x564179dbca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dbca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 -- 172.21.15.111:0/361105234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564179af8620 con 0x564179db4720 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4581000640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f4573e00640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179dbc530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f456402a940 tx=0x7f45640043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e36a54f12659341 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.133+0000 7f456a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564035040 con 0x564179db4720 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4574003dd0 con 0x564179db4720 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8557+0+0 (secure 0 0 0) 0x7f4564005450 con 0x564179db4720 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4564005650 con 0x564179db4720 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f4585e42740 10 client.25921 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564179dcdbc0 con 0x564179db4720 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.134+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574003dd0 con 0x564179db4720 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4564003960 con 0x564179db4720 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 10 client.25921 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 20 client.25921 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179dcf030 0x564179dd1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 4 client.25921 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564179d24ff0 con 0x564179dcf030 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.135+0000 7f4581000640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179dcf030 0x564179dd1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.136+0000 7f4581000640 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179dcf030 0x564179dd1420 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9218b298dec37c4 server_cookie=dcc5ce2c09b18faa in_seq=0 out_seq=0 2024-04-24T22:50:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.136+0000 7f456a000640 10 client.25921 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f456a000640 1 -- 172.21.15.111:0/361105234 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564179d24ff0 con 0x564179dcf030 2024-04-24T22:50:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f456a000640 10 client.25921 handle_command_reply: tid=0 2024-04-24T22:50:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 client.25921 shutdown 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 msgr2=0x564179dbc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179dbc530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f456402a940 tx=0x7f45640043b0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 shutdown_connections 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179dcf030 0x564179dd1420 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179dbc530 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 --2- 172.21.15.111:0/361105234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dbbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f456a000640 0 client.25921 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 >> 172.21.15.111:0/361105234 conn(0x564179db0310 msgr2=0x564179db6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.137+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 shutdown_connections 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.138+0000 7f4585e42740 1 -- 172.21.15.111:0/361105234 wait complete. 2024-04-24T22:50:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.138+0000 7f4585e42740 20 client.25921 trim_cache size 0 max 16384 2024-04-24T22:50:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 Processor -- start 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 -- start start 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dcaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179db5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1b1f60 con 0x564179dbefe0 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1af830 con 0x564179db3e00 2024-04-24T22:50:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4585e42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ad6a0 con 0x564179db4720 2024-04-24T22:50:08.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:07 smithi171 ceph-mon[32337]: pgmap v253: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:50:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4581000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4581000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51546/0 (socket says 172.21.15.111:51546) 2024-04-24T22:50:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4581000640 1 -- 172.21.15.111:0/2467997594 learned_addr learned my addr 172.21.15.111:0/2467997594 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4581000640 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dcaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.154+0000 7f4573e00640 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179db5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 -- 172.21.15.111:0/2467997594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 msgr2=0x564179db5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179db5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 -- 172.21.15.111:0/2467997594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 msgr2=0x564179dcaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dcaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 -- 172.21.15.111:0/2467997594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564179dab620 con 0x564179dbefe0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4581000640 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x564179dd1c60 tx=0x7f4574009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25008adff9396e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.155+0000 7f4570c00640 1 -- 172.21.15.111:0/2467997594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400f990 con 0x564179dbefe0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4570c00640 1 -- 172.21.15.111:0/2467997594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574003a30 con 0x564179dbefe0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4570c00640 1 -- 172.21.15.111:0/2467997594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574005800 con 0x564179dbefe0 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 -- 172.21.15.111:0/2467997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 msgr2=0x564179dd6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x564179dd1c60 tx=0x7f4574009500 comp rx=0 tx=0).stop 2024-04-24T22:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 -- 172.21.15.111:0/2467997594 shutdown_connections 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dd6cc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db4720 0x564179db5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 --2- 172.21.15.111:0/2467997594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db3e00 0x564179dcaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 -- 172.21.15.111:0/2467997594 >> 172.21.15.111:0/2467997594 conn(0x564179db0310 msgr2=0x564179db57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 -- 172.21.15.111:0/2467997594 shutdown_connections 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.156+0000 7f4585e42740 1 -- 172.21.15.111:0/2467997594 wait complete. 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 -- start start 2024-04-24T22:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 Processor -- start 2024-04-24T22:50:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179dc6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dc7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1adcf0 con 0x564179dbefe0 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1af160 con 0x564179db4720 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.157+0000 7f4585e42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ae3c0 con 0x564179db3e00 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4585e42740 10 client.0 _ll_register_callbacks cb 0x7f4564009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51552/0 (socket says 172.21.15.111:51552) 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 -- 172.21.15.111:0/1936385045 learned_addr learned my addr 172.21.15.111:0/1936385045 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 -- 172.21.15.111:0/1936385045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 msgr2=0x564179dc6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179dc6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 -- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 msgr2=0x564179dc7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dc7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 -- 172.21.15.111:0/1936385045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564179dc4640 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f4581000640 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4574011940 tx=0x7f457400f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84103369f7006da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.158+0000 7f456a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457403e030 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8557+0+0 (secure 0 0 0) 0x7f4574003d30 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4574003a30 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4574005030 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 10 client.15864 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564179e3e6e0 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574003a30 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4574028450 con 0x564179dbefe0 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 10 client.15864 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 20 client.15864 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179e3ebb0 0x564179e40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 4 client.15864 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.159+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564179d1f340 con 0x564179e3ebb0 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.160+0000 7f4573e00640 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179e3ebb0 0x564179e40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.160+0000 7f4573e00640 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179e3ebb0 0x564179e40fa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88bfb3c5a090d6b server_cookie=f60cdb1433d69ff3 in_seq=0 out_seq=0 2024-04-24T22:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.160+0000 7f456a000640 10 client.15864 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f456a000640 1 -- 172.21.15.111:0/1936385045 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564179d1f340 con 0x564179e3ebb0 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f456a000640 10 client.15864 handle_command_reply: tid=0 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 client.15864 shutdown 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 msgr2=0x564179dc3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4574011940 tx=0x7f457400f600 comp rx=0 tx=0).stop 2024-04-24T22:50:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 shutdown_connections 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x564179e3ebb0 0x564179e40fa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564179dbefe0 0x564179dc3fb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564179db4720 0x564179dc7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 --2- 172.21.15.111:0/1936385045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564179db3e00 0x564179dc6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f456a000640 0 client.15864 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 >> 172.21.15.111:0/1936385045 conn(0x564179db0310 msgr2=0x564179dbc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 shutdown_connections 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.174+0000 7f4585e42740 1 -- 172.21.15.111:0/1936385045 wait complete. 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4585e42740 20 client.15864 trim_cache size 0 max 16384 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:50:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 msgr2=0x7f45600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 0x7f45600a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ccfb0 tx=0x7f456c0d5550 comp rx=0 tx=0).stop 2024-04-24T22:50:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 msgr2=0x7f457c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9f0 tx=0x7f456c00edf0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 shutdown_connections 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f457c109950 0x7f457c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c104300 0x7f457c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f45600a2da0 0x7f45600a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 --2- 172.21.15.111:0/1754224627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f457c103940 0x7f457c19a270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.175+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 >> 172.21.15.111:0/1754224627 conn(0x7f457c0ff470 msgr2=0x7f457c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.176+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 shutdown_connections 2024-04-24T22:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:08.176+0000 7f4583000640 1 -- 172.21.15.111:0/1754224627 wait complete. 2024-04-24T22:50:08.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:07 smithi186 ceph-mon[32753]: pgmap v253: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:50:08.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:07 smithi111 ceph-mon[27746]: pgmap v253: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:50:10.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:09 smithi171 ceph-mon[32337]: pgmap v254: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:10.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:09 smithi186 ceph-mon[32753]: pgmap v254: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:10.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:09 smithi111 ceph-mon[27746]: pgmap v254: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:12.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:11 smithi171 ceph-mon[32337]: pgmap v255: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:12.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:11 smithi186 ceph-mon[32753]: pgmap v255: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:12.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:11 smithi111 ceph-mon[27746]: pgmap v255: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 32 op/s 2024-04-24T22:50:13.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:13 smithi186 ceph-mon[32753]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T22:50:13.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:13 smithi111 ceph-mon[27746]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T22:50:13.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:13 smithi171 ceph-mon[32337]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T22:50:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:14 smithi186 ceph-mon[32753]: pgmap v257: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:50:14.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:14 smithi111 ceph-mon[27746]: pgmap v257: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:50:14.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:14 smithi171 ceph-mon[32337]: pgmap v257: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:50:16.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:16 smithi186 ceph-mon[32753]: pgmap v258: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-04-24T22:50:16.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:16 smithi111 ceph-mon[27746]: pgmap v258: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-04-24T22:50:16.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:16 smithi171 ceph-mon[32337]: pgmap v258: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-04-24T22:50:17.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:17 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:17.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:17 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:17.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:17 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:18.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:18 smithi186 ceph-mon[32753]: pgmap v259: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:50:18.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:18 smithi111 ceph-mon[27746]: pgmap v259: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:50:18.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:18 smithi171 ceph-mon[32337]: pgmap v259: 97 pgs: 97 active+clean; 3.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:50:19.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:19 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:19.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:19 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:19.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:19 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:50:19.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:19 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:19.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:19 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:19.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:19 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:50:19.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:19 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:19.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:19 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:19.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:19 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:50:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:20 smithi111 ceph-mon[27746]: pgmap v260: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:20 smithi171 ceph-mon[32337]: pgmap v260: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:20.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:20 smithi186 ceph-mon[32753]: pgmap v260: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 -- 172.21.15.111:0/2623264303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314109950 msgr2=0x7f5314109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 --2- 172.21.15.111:0/2623264303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314109950 0x7f5314109d30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5304009930 tx=0x7f530402a420 comp rx=0 tx=0).stop 2024-04-24T22:50:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 -- 172.21.15.111:0/2623264303 shutdown_connections 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 --2- 172.21.15.111:0/2623264303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f5314110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 --2- 172.21.15.111:0/2623264303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314103940 0x7f5314103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 --2- 172.21.15.111:0/2623264303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314109950 0x7f5314109d30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 -- 172.21.15.111:0/2623264303 >> 172.21.15.111:0/2623264303 conn(0x7f53140ff470 msgr2=0x7f5314101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 -- 172.21.15.111:0/2623264303 shutdown_connections 2024-04-24T22:50:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.317+0000 7f531ba00640 1 -- 172.21.15.111:0/2623264303 wait complete. 2024-04-24T22:50:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.318+0000 7f531ba00640 1 Processor -- start 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 -- start start 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 0x7f531419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314109950 0x7f531419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5314113d30 con 0x7f5314103940 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5314113bb0 con 0x7f5314109950 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f531ba00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5314113eb0 con 0x7f5314104300 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5313e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5319000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 0x7f531419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5313e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:55834/0 (socket says 172.21.15.111:55834) 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5319a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314109950 0x7f531419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5313e00640 1 -- 172.21.15.111:0/3396904710 learned_addr learned my addr 172.21.15.111:0/3396904710 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.319+0000 7f5319000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 0x7f531419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58642/0 (socket says 172.21.15.111:58642) 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 -- 172.21.15.111:0/3396904710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314109950 msgr2=0x7f531419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314109950 0x7f531419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 -- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 msgr2=0x7f531419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 0x7f531419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 -- 172.21.15.111:0/3396904710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531419f250 con 0x7f5314104300 2024-04-24T22:50:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5313e00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00ebc0 tx=0x7f52fc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5311600640 1 -- 172.21.15.111:0/3396904710 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc010630 con 0x7f5314104300 2024-04-24T22:50:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5311600640 1 -- 172.21.15.111:0/3396904710 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc010cb0 con 0x7f5314104300 2024-04-24T22:50:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f5311600640 1 -- 172.21.15.111:0/3396904710 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc018bd0 con 0x7f5314104300 2024-04-24T22:50:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.320+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531419f540 con 0x7f5314104300 2024-04-24T22:50:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.321+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141a6dd0 con 0x7f5314104300 2024-04-24T22:50:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.324+0000 7f5311600640 1 -- 172.21.15.111:0/3396904710 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f52fc0107d0 con 0x7f5314104300 2024-04-24T22:50:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.324+0000 7f531e3fd740 1 Processor -- start 2024-04-24T22:50:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.324+0000 7f531e3fd740 1 -- start start 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563834ac0ab0 con 0x563834b5d720 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563834a69640 con 0x563834b5ce00 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f531e3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563834abca70 con 0x563834b67fe0 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58656/0 (socket says 172.21.15.111:58656) 2024-04-24T22:50:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 -- 172.21.15.111:0/4087907253 learned_addr learned my addr 172.21.15.111:0/4087907253 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5313e00640 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 -- 172.21.15.111:0/4087907253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 msgr2=0x563834b6a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b6a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 -- 172.21.15.111:0/4087907253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 msgr2=0x563834b5d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b5d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.325+0000 7f5319000640 1 -- 172.21.15.111:0/4087907253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638347d5210 con 0x563834b5d720 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f5311600640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 0x7f52f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f5319000640 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f530402a940 tx=0x7f5304005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e61e9dd633820a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f5311600640 1 -- 172.21.15.111:0/3396904710 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f52fc0cfad0 con 0x7f5314104300 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f5319a00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 0x7f52f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f530a000640 1 -- 172.21.15.111:0/4087907253 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304040070 con 0x563834b5d720 2024-04-24T22:50:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f530a000640 1 -- 172.21.15.111:0/4087907253 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304031870 con 0x563834b5d720 2024-04-24T22:50:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.326+0000 7f530a000640 1 -- 172.21.15.111:0/4087907253 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304035040 con 0x563834b5d720 2024-04-24T22:50:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 -- 172.21.15.111:0/4087907253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 msgr2=0x563834b67aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f5319a00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 0x7f52f80a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f530c009d20 tx=0x7f530c005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f530402a940 tx=0x7f5304005bd0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 -- 172.21.15.111:0/4087907253 shutdown_connections 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b67aa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 --2- 172.21.15.111:0/4087907253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b5d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 -- 172.21.15.111:0/4087907253 >> 172.21.15.111:0/4087907253 conn(0x563834b59310 msgr2=0x563834b596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 -- 172.21.15.111:0/4087907253 shutdown_connections 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.327+0000 7f531e3fd740 1 -- 172.21.15.111:0/4087907253 wait complete. 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 -- start start 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 Processor -- start 2024-04-24T22:50:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563834b6abb0 con 0x563834b5d720 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563834a7c990 con 0x563834b5ce00 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563834b6ad30 con 0x563834b67fe0 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.328+0000 7f531e3fd740 10 client.0 _ll_register_callbacks cb 0x563834a7e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5313e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:55854/0 (socket says 172.21.15.111:55854) 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b64f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:35264/0 (socket says 172.21.15.111:35264) 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 -- 172.21.15.111:0/3639597524 learned_addr learned my addr 172.21.15.111:0/3639597524 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 -- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 msgr2=0x563834b64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 -- 172.21.15.111:0/3639597524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 msgr2=0x563834b654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 -- 172.21.15.111:0/3639597524 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638348a1620 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5319a00640 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f530c0045e0 tx=0x7f530c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82db242576d9f51c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.329+0000 7f5302a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c01e470 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f530c01e610 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5304031870 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f530c01e810 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f531e3fd740 10 client.25936 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563834b76b50 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304031870 con 0x563834b67fe0 2024-04-24T22:50:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.330+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f530c027070 con 0x563834b67fe0 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 10 client.25936 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 20 client.25936 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834b78060 0x563834b7a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 4 client.25936 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563834acdff0 con 0x563834b78060 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.331+0000 7f5319000640 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834b78060 0x563834b7a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.332+0000 7f5302a00640 10 client.25936 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.332+0000 7f5319000640 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834b78060 0x563834b7a470 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18bac7eaba8518e server_cookie=ab7693463da6214d in_seq=0 out_seq=0 2024-04-24T22:50:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.332+0000 7f5302a00640 1 -- 172.21.15.111:0/3639597524 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563834acdff0 con 0x563834b78060 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.332+0000 7f5302a00640 10 client.25936 handle_command_reply: tid=0 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.332+0000 7f531e3fd740 1 client.25936 shutdown 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 msgr2=0x563834b659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f530c0045e0 tx=0x7f530c004730 comp rx=0 tx=0).stop 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 shutdown_connections 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834b78060 0x563834b7a470 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b67fe0 0x563834b659e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b5d720 0x563834b654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3639597524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f5302a00640 0 client.25936 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 >> 172.21.15.111:0/3639597524 conn(0x563834b59310 msgr2=0x563834b5f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 shutdown_connections 2024-04-24T22:50:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 1 -- 172.21.15.111:0/3639597524 wait complete. 2024-04-24T22:50:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.333+0000 7f531e3fd740 20 client.25936 trim_cache size 0 max 16384 2024-04-24T22:50:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 Processor -- start 2024-04-24T22:50:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 -- start start 2024-04-24T22:50:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b58300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f53141b1ed0 con 0x563834b67fe0 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53141af7a0 con 0x563834b5ce00 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.349+0000 7f531e3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f53141ad610 con 0x563834b5d720 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5319a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b58300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:55856/0 (socket says 172.21.15.111:55856) 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5319000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 -- 172.21.15.111:0/3206253052 learned_addr learned my addr 172.21.15.111:0/3206253052 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 -- 172.21.15.111:0/3206253052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 msgr2=0x563834b73a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b73a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 -- 172.21.15.111:0/3206253052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 msgr2=0x563834b58300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b58300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 -- 172.21.15.111:0/3206253052 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563834b54620 con 0x563834b5d720 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5319a00640 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b58300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5313e00640 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52fc09bd40 tx=0x7f52fc0d3580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08e0803b72be465 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.350+0000 7f5309600640 1 -- 172.21.15.111:0/3206253052 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc010e60 con 0x563834b5d720 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f5309600640 1 -- 172.21.15.111:0/3206253052 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc09c040 con 0x563834b5d720 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f5309600640 1 -- 172.21.15.111:0/3206253052 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc010ac0 con 0x563834b5d720 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 -- 172.21.15.111:0/3206253052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 msgr2=0x563834b5f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52fc09bd40 tx=0x7f52fc0d3580 comp rx=0 tx=0).stop 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 -- 172.21.15.111:0/3206253052 shutdown_connections 2024-04-24T22:50:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b58300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b5f810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 --2- 172.21.15.111:0/3206253052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b73a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 -- 172.21.15.111:0/3206253052 >> 172.21.15.111:0/3206253052 conn(0x563834b59310 msgr2=0x563834b57bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 -- 172.21.15.111:0/3206253052 shutdown_connections 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.351+0000 7f531e3fd740 1 -- 172.21.15.111:0/3206253052 wait complete. 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 -- start start 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 Processor -- start 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b6ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f53141adc60 con 0x563834b67fe0 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53141af0d0 con 0x563834b5ce00 2024-04-24T22:50:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.352+0000 7f531e3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f53141ae330 con 0x563834b5d720 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f531e3fd740 10 client.0 _ll_register_callbacks cb 0x7f530c009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b6ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58674/0 (socket says 172.21.15.111:58674) 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 -- 172.21.15.111:0/2628038090 learned_addr learned my addr 172.21.15.111:0/2628038090 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5313e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 -- 172.21.15.111:0/2628038090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 msgr2=0x563834b704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 -- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 msgr2=0x563834b6ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b6ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 -- 172.21.15.111:0/2628038090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563834b5fd50 con 0x563834b67fe0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5319a00640 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f530c029d20 tx=0x7f530c01e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cccd65dc4e880c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.353+0000 7f5302a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:50:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c0043d0 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f530c01ebc0 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc09c040 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f530c040050 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 10 client.15882 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563834bded10 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc09c040 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f530c038020 con 0x563834b67fe0 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 10 client.15882 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 20 client.15882 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.354+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834bdf4e0 0x563834be1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.355+0000 7f531e3fd740 4 client.15882 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:50:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.355+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563834ac8340 con 0x563834bdf4e0 2024-04-24T22:50:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.355+0000 7f5319000640 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834bdf4e0 0x563834be1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.355+0000 7f5319000640 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834bdf4e0 0x563834be1900 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e34b310b31a19f server_cookie=8ded8b3e94ed98b1 in_seq=0 out_seq=0 2024-04-24T22:50:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.355+0000 7f5302a00640 10 client.15882 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.369+0000 7f5302a00640 1 -- 172.21.15.111:0/2628038090 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563834ac8340 con 0x563834bdf4e0 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.369+0000 7f5302a00640 10 client.15882 handle_command_reply: tid=0 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.369+0000 7f531e3fd740 1 client.15882 shutdown 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 msgr2=0x563834b6cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f530c029d20 tx=0x7f530c01e720 comp rx=0 tx=0).stop 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 shutdown_connections 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x563834bdf4e0 0x563834be1900 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563834b67fe0 0x563834b6cf50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563834b5d720 0x563834b704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 --2- 172.21.15.111:0/2628038090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563834b5ce00 0x563834b6ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f5302a00640 0 client.15882 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 >> 172.21.15.111:0/2628038090 conn(0x563834b59310 msgr2=0x563834b58480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 shutdown_connections 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 1 -- 172.21.15.111:0/2628038090 wait complete. 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.370+0000 7f531e3fd740 20 client.15882 trim_cache size 0 max 16384 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:50:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 msgr2=0x7f52f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 0x7f52f80a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f530c009d20 tx=0x7f530c005f90 comp rx=0 tx=0).stop 2024-04-24T22:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 msgr2=0x7f531419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00ebc0 tx=0x7f52fc00c6a0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 shutdown_connections 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5314109950 0x7f531419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f52f80a2da0 0x7f52f80a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5314104300 0x7f531419a7f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 --2- 172.21.15.111:0/3396904710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5314103940 0x7f531419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 >> 172.21.15.111:0/3396904710 conn(0x7f53140ff470 msgr2=0x7f53140ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 shutdown_connections 2024-04-24T22:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:21.371+0000 7f531ba00640 1 -- 172.21.15.111:0/3396904710 wait complete. 2024-04-24T22:50:22.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:22 smithi111 ceph-mon[27746]: pgmap v261: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:22 smithi171 ceph-mon[32337]: pgmap v261: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:22.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:22 smithi186 ceph-mon[32753]: pgmap v261: 97 pgs: 97 active+clean; 3.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-04-24T22:50:24.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:24 smithi111 ceph-mon[27746]: pgmap v262: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-04-24T22:50:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:24 smithi171 ceph-mon[32337]: pgmap v262: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-04-24T22:50:24.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:24 smithi186 ceph-mon[32753]: pgmap v262: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-04-24T22:50:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:26 smithi171 ceph-mon[32337]: pgmap v263: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:26.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:26 smithi186 ceph-mon[32753]: pgmap v263: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:26 smithi111 ceph-mon[27746]: pgmap v263: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:28.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:28 smithi171 ceph-mon[32337]: pgmap v264: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:28.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:28 smithi186 ceph-mon[32753]: pgmap v264: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:28.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:28 smithi111 ceph-mon[27746]: pgmap v264: 97 pgs: 97 active+clean; 3.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:50:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:30 smithi171 ceph-mon[32337]: pgmap v265: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:50:30.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:30 smithi186 ceph-mon[32753]: pgmap v265: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:50:30.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:30 smithi111 ceph-mon[27746]: pgmap v265: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:50:32.587 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:32 smithi111 ceph-mon[27746]: pgmap v266: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:32 smithi171 ceph-mon[32337]: pgmap v266: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:32 smithi186 ceph-mon[32753]: pgmap v266: 97 pgs: 97 active+clean; 4.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 -- 172.21.15.111:0/655558418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101570 msgr2=0x7f16c01019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 --2- 172.21.15.111:0/655558418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101570 0x7f16c01019f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009890 tx=0x7f16ac02a370 comp rx=0 tx=0).stop 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 -- 172.21.15.111:0/655558418 shutdown_connections 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 --2- 172.21.15.111:0/655558418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 0x7f16c010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 --2- 172.21.15.111:0/655558418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101570 0x7f16c01019f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 --2- 172.21.15.111:0/655558418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0107630 0x7f16c0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 -- 172.21.15.111:0/655558418 >> 172.21.15.111:0/655558418 conn(0x7f16c00fd080 msgr2=0x7f16c00ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 -- 172.21.15.111:0/655558418 shutdown_connections 2024-04-24T22:50:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.513+0000 7f16c5e00640 1 -- 172.21.15.111:0/655558418 wait complete. 2024-04-24T22:50:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.514+0000 7f16c5e00640 1 Processor -- start 2024-04-24T22:50:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 -- start start 2024-04-24T22:50:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 0x7f16c0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0107630 0x7f16c019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01118f0 con 0x7f16c0101f30 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16c0111770 con 0x7f16c0101570 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16c0111a70 con 0x7f16c0107630 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43782/0 (socket says 172.21.15.111:43782) 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c4a00640 1 -- 172.21.15.111:0/1921892535 learned_addr learned my addr 172.21.15.111:0/1921892535 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16bfe00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 0x7f16c0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.515+0000 7f16c5400640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0107630 0x7f16c019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 -- 172.21.15.111:0/1921892535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0107630 msgr2=0x7f16c019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0107630 0x7f16c019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 -- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 msgr2=0x7f16c0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 0x7f16c0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 -- 172.21.15.111:0/1921892535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c019ce00 con 0x7f16c0101570 2024-04-24T22:50:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16c4a00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d950 tx=0x7f16b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16bd600640 1 -- 172.21.15.111:0/1921892535 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4017030 con 0x7f16c0101570 2024-04-24T22:50:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.516+0000 7f16bd600640 1 -- 172.21.15.111:0/1921892535 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b4004020 con 0x7f16c0101570 2024-04-24T22:50:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.517+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c019d0f0 con 0x7f16c0101570 2024-04-24T22:50:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.517+0000 7f16bd600640 1 -- 172.21.15.111:0/1921892535 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40047a0 con 0x7f16c0101570 2024-04-24T22:50:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.517+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c01a4980 con 0x7f16c0101570 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 Processor -- start 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 -- start start 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16bd600640 1 -- 172.21.15.111:0/1921892535 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f16b40041d0 con 0x7f16c0101570 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bb6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb67b0 0x5639d6bc5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5639d6b19ab0 con 0x5639d6bc55a0 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5639d6ac2640 con 0x5639d6bb5e90 2024-04-24T22:50:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.520+0000 7f16ca226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639d6b15a70 con 0x5639d6bb67b0 2024-04-24T22:50:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41286/0 (socket says 172.21.15.111:41286) 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 -- 172.21.15.111:0/2411975553 learned_addr learned my addr 172.21.15.111:0/2411975553 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16c5400640 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bb6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16c4a00640 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb67b0 0x5639d6bc5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 -- 172.21.15.111:0/2411975553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb67b0 msgr2=0x5639d6bc5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb67b0 0x5639d6bc5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 -- 172.21.15.111:0/2411975553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 msgr2=0x5639d6bb6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bb6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.521+0000 7f16bfe00640 1 -- 172.21.15.111:0/2411975553 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d682e210 con 0x5639d6bc55a0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009890 tx=0x7f16ac005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69076808a6621fef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16b1c00640 1 -- 172.21.15.111:0/2411975553 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037040 con 0x5639d6bc55a0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16b1c00640 1 -- 172.21.15.111:0/2411975553 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac042070 con 0x5639d6bc55a0 2024-04-24T22:50:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16b1c00640 1 -- 172.21.15.111:0/2411975553 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac003880 con 0x5639d6bc55a0 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16bd600640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 0x7f16a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16bd600640 1 -- 172.21.15.111:0/1921892535 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f16b40ced90 con 0x7f16c0101570 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.522+0000 7f16c5400640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 0x7f16a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 -- 172.21.15.111:0/2411975553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 msgr2=0x5639d6bc7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009890 tx=0x7f16ac005b20 comp rx=0 tx=0).stop 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 -- 172.21.15.111:0/2411975553 shutdown_connections 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6bc7990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb67b0 0x5639d6bc5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 --2- 172.21.15.111:0/2411975553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bb6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 -- 172.21.15.111:0/2411975553 >> 172.21.15.111:0/2411975553 conn(0x5639d6bb24c0 msgr2=0x5639d6bb28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 -- 172.21.15.111:0/2411975553 shutdown_connections 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16ca226740 1 -- 172.21.15.111:0/2411975553 wait complete. 2024-04-24T22:50:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.523+0000 7f16c5400640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 0x7f16a80a5230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005fd0 tx=0x7f16b8010040 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 -- start start 2024-04-24T22:50:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 Processor -- start 2024-04-24T22:50:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5639d6bc8170 con 0x5639d6bb5e90 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5639d6ad5990 con 0x5639d6bb67b0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639d6bc82f0 con 0x5639d6bc55a0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.524+0000 7f16ca226740 10 client.0 _ll_register_callbacks cb 0x5639d6ad7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41290/0 (socket says 172.21.15.111:41290) 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c4a00640 1 -- 172.21.15.111:0/2877408624 learned_addr learned my addr 172.21.15.111:0/2877408624 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 -- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 msgr2=0x5639d6bc2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 -- 172.21.15.111:0/2877408624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 msgr2=0x5639d6bc2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 -- 172.21.15.111:0/2877408624 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d68fa620 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c4a00640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16c5400640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bc2f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16b8004390 tx=0x7f16b8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed1f5adb96b2ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.525+0000 7f16a6000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8020040 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f16b8004b20 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ac042070 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f16b8004850 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16ca226740 10 client.25954 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5639d6bd4110 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac042070 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.526+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f16b800b040 con 0x5639d6bc55a0 2024-04-24T22:50:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 10 client.25954 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 20 client.25954 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6bd55d0 0x5639d6bd79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 4 client.25954 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639d6b26ff0 con 0x5639d6bd55d0 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16c4a00640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6bd55d0 0x5639d6bd79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16c4a00640 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6bd55d0 0x5639d6bd79c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b69ca330f25c97 server_cookie=1f5734a82f27d402 in_seq=0 out_seq=0 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.527+0000 7f16a6000640 10 client.25954 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.528+0000 7f16a6000640 1 -- 172.21.15.111:0/2877408624 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639d6b26ff0 con 0x5639d6bd55d0 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.528+0000 7f16a6000640 10 client.25954 handle_command_reply: tid=0 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.528+0000 7f16ca226740 1 client.25954 shutdown 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.528+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 msgr2=0x5639d6bc2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.528+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bc2f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16b8004390 tx=0x7f16b8004ab0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 shutdown_connections 2024-04-24T22:50:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6bd55d0 0x5639d6bd79c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bc2f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16a6000640 0 client.25954 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 --2- 172.21.15.111:0/2877408624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb5e90 0x5639d6bc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 >> 172.21.15.111:0/2877408624 conn(0x5639d6bb24c0 msgr2=0x5639d6bb8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 shutdown_connections 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 1 -- 172.21.15.111:0/2877408624 wait complete. 2024-04-24T22:50:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.529+0000 7f16ca226740 20 client.25954 trim_cache size 0 max 16384 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.545+0000 7f16ca226740 1 Processor -- start 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 -- start start 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 0x5639d6bdd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bdd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01ad960 con 0x5639d6bb67b0 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01ab230 con 0x5639d6bb5e90 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16ca226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01a90a0 con 0x5639d6bc55a0 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16c5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bdd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bd10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43802/0 (socket says 172.21.15.111:43802) 2024-04-24T22:50:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16c4a00640 1 -- 172.21.15.111:0/2341202542 learned_addr learned my addr 172.21.15.111:0/2341202542 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.546+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 0x5639d6bdd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 -- 172.21.15.111:0/2341202542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 msgr2=0x5639d6bd10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bd10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 -- 172.21.15.111:0/2341202542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 msgr2=0x5639d6bdd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 0x5639d6bdd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 -- 172.21.15.111:0/2341202542 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d6bb65e0 con 0x5639d6bc55a0 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16bfe00640 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 0x5639d6bdd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16c5400640 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bdd7d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b801af40 tx=0x7f16b8009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43e9e722531ef9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16b1200640 1 -- 172.21.15.111:0/2341202542 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b800d8d0 con 0x5639d6bc55a0 2024-04-24T22:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16b1200640 1 -- 172.21.15.111:0/2341202542 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b80247c0 con 0x5639d6bc55a0 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.547+0000 7f16b1200640 1 -- 172.21.15.111:0/2341202542 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b801a6b0 con 0x5639d6bc55a0 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 -- 172.21.15.111:0/2341202542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 msgr2=0x5639d6bdd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bdd7d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b801af40 tx=0x7f16b8009460 comp rx=0 tx=0).stop 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 -- 172.21.15.111:0/2341202542 shutdown_connections 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bc55a0 0x5639d6bdd7d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bb67b0 0x5639d6bdd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 --2- 172.21.15.111:0/2341202542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb5e90 0x5639d6bd10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 -- 172.21.15.111:0/2341202542 >> 172.21.15.111:0/2341202542 conn(0x5639d6bb24c0 msgr2=0x5639d6bb28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 -- 172.21.15.111:0/2341202542 shutdown_connections 2024-04-24T22:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.548+0000 7f16ca226740 1 -- 172.21.15.111:0/2341202542 wait complete. 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.549+0000 7f16ca226740 1 -- start start 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.549+0000 7f16ca226740 1 Processor -- start 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6c3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01a96f0 con 0x5639d6bc55a0 2024-04-24T22:50:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01aab60 con 0x5639d6bb67b0 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01a9dc0 con 0x5639d6bb5e90 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16ca226740 10 client.0 _ll_register_callbacks cb 0x7f16b801d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52370/0 (socket says 172.21.15.111:52370) 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 -- 172.21.15.111:0/3779747898 learned_addr learned my addr 172.21.15.111:0/3779747898 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:43808/0 (socket says 172.21.15.111:43808) 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c5400640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6c3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 -- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 msgr2=0x5639d6bc9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 -- 172.21.15.111:0/3779747898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 msgr2=0x5639d6c3ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6c3ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16c4a00640 1 -- 172.21.15.111:0/3779747898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d6c3b040 con 0x5639d6bb5e90 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.550+0000 7f16bfe00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16c4a00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f16b40976b0 tx=0x7f16b40bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa16e160e01e0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b409abb0 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b80247c0 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f16b40cc700 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f16b40cc900 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16ca226740 10 client.25960 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5639d6c3c250 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b80247c0 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.551+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f16b40a3a30 con 0x5639d6bb5e90 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 10 client.25960 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 20 client.25960 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6c3c920 0x5639d6c3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 4 client.25960 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:50:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639d6b21340 con 0x5639d6c3c920 2024-04-24T22:50:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.552+0000 7f16bfe00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6c3c920 0x5639d6c3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.553+0000 7f16bfe00640 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6c3c920 0x5639d6c3ed10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d3c49f62d4cb9f server_cookie=84ca8fbb0a1c90cf in_seq=0 out_seq=0 2024-04-24T22:50:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.553+0000 7f16a6000640 10 client.25960 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16a6000640 1 -- 172.21.15.111:0/3779747898 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639d6b21340 con 0x5639d6c3c920 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16a6000640 10 client.25960 handle_command_reply: tid=0 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 client.25960 shutdown 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 msgr2=0x5639d6bc93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f16b40976b0 tx=0x7f16b40bb040 comp rx=0 tx=0).stop 2024-04-24T22:50:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 shutdown_connections 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x5639d6c3c920 0x5639d6c3ed10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639d6bc55a0 0x5639d6c3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639d6bb67b0 0x5639d6bc9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 --2- 172.21.15.111:0/3779747898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639d6bb5e90 0x5639d6bc93c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16a6000640 0 client.25960 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 >> 172.21.15.111:0/3779747898 conn(0x5639d6bb24c0 msgr2=0x5639d6bb0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 shutdown_connections 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 1 -- 172.21.15.111:0/3779747898 wait complete. 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.568+0000 7f16ca226740 20 client.25960 trim_cache size 0 max 16384 2024-04-24T22:50:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:50:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:50:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:50:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:50:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 msgr2=0x7f16a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 0x7f16a80a5230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005fd0 tx=0x7f16b8010040 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 msgr2=0x7f16c0197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d950 tx=0x7f16b400de10 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 shutdown_connections 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f16a80a2d70 0x7f16a80a5230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0107630 0x7f16c019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16c0101f30 0x7f16c0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 --2- 172.21.15.111:0/1921892535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16c0101570 0x7f16c0197e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 >> 172.21.15.111:0/1921892535 conn(0x7f16c00fd080 msgr2=0x7f16c00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.569+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 shutdown_connections 2024-04-24T22:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:34.570+0000 7f16c5e00640 1 -- 172.21.15.111:0/1921892535 wait complete. 2024-04-24T22:50:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:34 smithi171 ceph-mon[32337]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:34 smithi186 ceph-mon[32753]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:34.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:34 smithi111 ceph-mon[27746]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:36.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:36 smithi171 ceph-mon[32337]: pgmap v268: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:36.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:36 smithi186 ceph-mon[32753]: pgmap v268: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:36.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:36 smithi111 ceph-mon[27746]: pgmap v268: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:38 smithi171 ceph-mon[32337]: pgmap v269: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:38 smithi186 ceph-mon[32753]: pgmap v269: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:38.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:38 smithi111 ceph-mon[27746]: pgmap v269: 97 pgs: 97 active+clean; 4.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 9 op/s 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:375/600: open fd = 3 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:376/600: open fd = 3 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:377/600: open fd = 3 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.611 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:378/600: open fd = 3 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:379/600: open fd = 3 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:380/600: open fd = 3 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:381/600: open fd = 3 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.612 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:382/600: open fd = 3 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:383/600: open fd = 3 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:384/600: open fd = 3 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:385/600: open fd = 3 2024-04-24T22:50:39.613 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:386/600: open fd = 3 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:387/600: open fd = 3 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:388/600: open fd = 3 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.614 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:389/600: open fd = 3 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:390/600: open fd = 3 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:391/600: open fd = 3 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:392/600: open fd = 3 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.615 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:393/600: open fd = 3 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:394/600: open fd = 3 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.616 INFO:tasks.workunit.client.0.smithi111.stdout:395/600: open fd = 3 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:396/600: open fd = 3 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:397/600: open fd = 3 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:398/600: open fd = 3 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.617 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:399/600: open fd = 3 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:400/600: open fd = 3 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:401/600: open fd = 3 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:402/600: open fd = 3 2024-04-24T22:50:39.618 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:403/600: open fd = 3 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:404/600: open fd = 3 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:405/600: open fd = 3 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:406/600: open fd = 3 2024-04-24T22:50:39.619 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:407/600: open fd = 3 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:408/600: open fd = 3 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:409/600: open fd = 3 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.620 INFO:tasks.workunit.client.0.smithi111.stdout:410/600: open fd = 3 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:411/600: open fd = 3 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:412/600: open fd = 3 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:413/600: open fd = 3 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.621 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:414/600: open fd = 3 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:415/600: open fd = 3 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:416/600: open fd = 3 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:417/600: open fd = 3 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.622 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:418/600: open fd = 3 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:419/600: open fd = 3 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:420/600: open fd = 3 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:421/600: open fd = 3 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.623 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:422/600: open fd = 3 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:423/600: open fd = 3 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:424/600: open fd = 3 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:425/600: open fd = 3 2024-04-24T22:50:39.624 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:426/600: open fd = 3 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:427/600: open fd = 3 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:428/600: open fd = 3 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.625 INFO:tasks.workunit.client.0.smithi111.stdout:429/600: open fd = 3 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:430/600: open fd = 3 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:431/600: open fd = 3 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:432/600: open fd = 3 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.626 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:433/600: open fd = 3 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:434/600: open fd = 3 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:435/600: open fd = 3 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:436/600: open fd = 3 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:50:39.627 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:50:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:40 smithi171 ceph-mon[32337]: pgmap v270: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:40 smithi186 ceph-mon[32753]: pgmap v270: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:40.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:40 smithi111 ceph-mon[27746]: pgmap v270: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-24T22:50:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:42 smithi171 ceph-mon[32337]: pgmap v271: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-24T22:50:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:42 smithi186 ceph-mon[32753]: pgmap v271: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-24T22:50:42.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:42 smithi111 ceph-mon[27746]: pgmap v271: 97 pgs: 97 active+clean; 4.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-24T22:50:44.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:44 smithi171 ceph-mon[32337]: pgmap v272: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:50:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:44 smithi186 ceph-mon[32753]: pgmap v272: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:50:44.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:44 smithi111 ceph-mon[27746]: pgmap v272: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:50:46.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:46 smithi171 ceph-mon[32337]: pgmap v273: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:46 smithi186 ceph-mon[32753]: pgmap v273: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:46.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:46 smithi111 ceph-mon[27746]: pgmap v273: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 -- 172.21.15.111:0/1332496772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 msgr2=0x7f7304109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 --2- 172.21.15.111:0/1332496772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f7304109d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009890 tx=0x7f72f402a380 comp rx=0 tx=0).stop 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 -- 172.21.15.111:0/1332496772 shutdown_connections 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 --2- 172.21.15.111:0/1332496772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f7304110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 --2- 172.21.15.111:0/1332496772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 0x7f7304103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 --2- 172.21.15.111:0/1332496772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f7304109d30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 -- 172.21.15.111:0/1332496772 >> 172.21.15.111:0/1332496772 conn(0x7f73040ff410 msgr2=0x7f7304101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 -- 172.21.15.111:0/1332496772 shutdown_connections 2024-04-24T22:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.712+0000 7f730b200640 1 -- 172.21.15.111:0/1332496772 wait complete. 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 Processor -- start 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 -- start start 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 0x7f730419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f730419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7304113d60 con 0x7f7304103940 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7304113be0 con 0x7f7304104300 2024-04-24T22:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.713+0000 7f730b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7304113ee0 con 0x7f7304109950 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 0x7f730419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:55848/0 (socket says 172.21.15.111:55848) 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 -- 172.21.15.111:0/2176557673 learned_addr learned my addr 172.21.15.111:0/2176557673 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f7309200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f730419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 -- 172.21.15.111:0/2176557673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 msgr2=0x7f730419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f730419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 -- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 msgr2=0x7f730419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 0x7f730419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.714+0000 7f72fbe00640 1 -- 172.21.15.111:0/2176557673 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730419f1d0 con 0x7f7304104300 2024-04-24T22:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f72fbe00640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f7301600640 1 -- 172.21.15.111:0/2176557673 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00cd80 con 0x7f7304104300 2024-04-24T22:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f7301600640 1 -- 172.21.15.111:0/2176557673 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec010910 con 0x7f7304104300 2024-04-24T22:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730419f4c0 con 0x7f7304104300 2024-04-24T22:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f7301600640 1 -- 172.21.15.111:0/2176557673 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018990 con 0x7f7304104300 2024-04-24T22:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.715+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73041a6d50 con 0x7f7304104300 2024-04-24T22:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.718+0000 7f730e1ad740 1 Processor -- start 2024-04-24T22:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.718+0000 7f730e1ad740 1 -- start start 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c031683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c030beab0 con 0x560c0315b720 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c03067640 con 0x560c0315ae00 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f730e1ad740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c030baa70 con 0x560c03165fe0 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:55852/0 (socket says 172.21.15.111:55852) 2024-04-24T22:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 -- 172.21.15.111:0/3869874224 learned_addr learned my addr 172.21.15.111:0/3869874224 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 -- 172.21.15.111:0/3869874224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 msgr2=0x560c031683d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c031683d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 -- 172.21.15.111:0/3869874224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 msgr2=0x560c03165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.719+0000 7f7309200640 1 -- 172.21.15.111:0/3869874224 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c02dd3210 con 0x560c0315ae00 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.720+0000 7f7301600640 1 -- 172.21.15.111:0/2176557673 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f72ec010ac0 con 0x7f7304104300 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.720+0000 7f7309200640 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e480 tx=0x7f72fc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa4077f03417adf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.720+0000 7f72f9600640 1 -- 172.21.15.111:0/3869874224 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc003970 con 0x560c0315ae00 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.720+0000 7f72f9600640 1 -- 172.21.15.111:0/3869874224 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72fc00d7d0 con 0x560c0315ae00 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.720+0000 7f72f9600640 1 -- 172.21.15.111:0/3869874224 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0044a0 con 0x560c0315ae00 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 -- 172.21.15.111:0/3869874224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 msgr2=0x560c0315b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e480 tx=0x7f72fc0095c0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 -- 172.21.15.111:0/3869874224 shutdown_connections 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c031683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3869874224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315ae00 0x560c0315b1e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 -- 172.21.15.111:0/3869874224 >> 172.21.15.111:0/3869874224 conn(0x560c03157310 msgr2=0x560c031576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 -- 172.21.15.111:0/3869874224 shutdown_connections 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f730e1ad740 1 -- 172.21.15.111:0/3869874224 wait complete. 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.721+0000 7f7301600640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 0x7f72e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f7301600640 1 -- 172.21.15.111:0/2176557673 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f72ec0cf7a0 con 0x7f7304104300 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 -- start start 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f7303e00640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 0x7f72e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 Processor -- start 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 0x560c03163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c03163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c03168c10 con 0x560c0315b720 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c0307a990 con 0x560c03165fe0 2024-04-24T22:50:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.722+0000 7f730e1ad740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c03168d90 con 0x560c0315ae00 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7303e00640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 0x7f72e80a5170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f72f402a970 tx=0x7f72f40056d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f730e1ad740 10 client.0 _ll_register_callbacks cb 0x560c0307c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7303e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 0x560c03163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7303e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 0x560c03163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:43396/0 (socket says 172.21.15.111:43396) 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7303e00640 1 -- 172.21.15.111:0/3367502705 learned_addr learned my addr 172.21.15.111:0/3367502705 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7309200640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c03163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 -- 172.21.15.111:0/3367502705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 msgr2=0x560c03163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 0x560c03163010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 -- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 msgr2=0x560c03163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c03163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 -- 172.21.15.111:0/3367502705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c02e9f620 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f7309200640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c03163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72fbe00640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03163550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0cce90 tx=0x7f72ec09b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8953be1b8cbc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.723+0000 7f72f2000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018d30 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f72ec09b160 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72fc00d7d0 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72ec0b9050 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 10 client.15912 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560c03174bd0 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72fc00d7d0 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72ec010550 con 0x560c0315b720 2024-04-24T22:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.724+0000 7f730e1ad740 20 client.15912 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c03176040 0x560c03178430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f730e1ad740 4 client.15912 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c030cbff0 con 0x560c03176040 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f7303e00640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c03176040 0x560c03178430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f7303e00640 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c03176040 0x560c03178430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4410eac35506f1c8 server_cookie=3dcbcf3327f20213 in_seq=0 out_seq=0 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.725+0000 7f72f2000640 10 client.15912 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f72f2000640 1 -- 172.21.15.111:0/3367502705 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c030cbff0 con 0x560c03176040 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f72f2000640 10 client.15912 handle_command_reply: tid=0 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 client.15912 shutdown 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 msgr2=0x560c03163550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03163550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0cce90 tx=0x7f72ec09b940 comp rx=0 tx=0).stop 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 shutdown_connections 2024-04-24T22:50:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c03176040 0x560c03178430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c03163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315b720 0x560c03163550 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 --2- 172.21.15.111:0/3367502705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315ae00 0x560c03163010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f72f2000640 0 client.15912 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.726+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 >> 172.21.15.111:0/3367502705 conn(0x560c03157310 msgr2=0x560c0315d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.727+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 shutdown_connections 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.727+0000 7f730e1ad740 1 -- 172.21.15.111:0/3367502705 wait complete. 2024-04-24T22:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.727+0000 7f730e1ad740 20 client.15912 trim_cache size 0 max 16384 2024-04-24T22:50:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 Processor -- start 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 -- start start 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 0x560c0317dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c0317e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73041b1f70 con 0x560c0315ae00 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f73041af840 con 0x560c0315b720 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f730e1ad740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f73041ad6b0 con 0x560c03165fe0 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37820/0 (socket says 172.21.15.111:37820) 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.743+0000 7f7303e00640 1 -- 172.21.15.111:0/4016951919 learned_addr learned my addr 172.21.15.111:0/4016951919 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f72fbe00640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 0x560c0317dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7309200640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c0317e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 -- 172.21.15.111:0/4016951919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 msgr2=0x560c0317e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c0317e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 -- 172.21.15.111:0/4016951919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 msgr2=0x560c0317dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 0x560c0317dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 -- 172.21.15.111:0/4016951919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c03157ee0 con 0x560c0315ae00 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7309200640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c0317e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f72fbe00640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 0x560c0317dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:50:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f7303e00640 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009860 tx=0x7f72f403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a88461cc762622 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f72f8c00640 1 -- 172.21.15.111:0/4016951919 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4037040 con 0x560c0315ae00 2024-04-24T22:50:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f72f8c00640 1 -- 172.21.15.111:0/4016951919 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4057070 con 0x560c0315ae00 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.744+0000 7f72f8c00640 1 -- 172.21.15.111:0/4016951919 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4003b20 con 0x560c0315ae00 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 -- 172.21.15.111:0/4016951919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 msgr2=0x560c03171ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009860 tx=0x7f72f403d000 comp rx=0 tx=0).stop 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 -- 172.21.15.111:0/4016951919 shutdown_connections 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c03165fe0 0x560c0317e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c0315b720 0x560c0317dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 --2- 172.21.15.111:0/4016951919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c03171ad0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 -- 172.21.15.111:0/4016951919 >> 172.21.15.111:0/4016951919 conn(0x560c03157310 msgr2=0x560c03156480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 -- 172.21.15.111:0/4016951919 shutdown_connections 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.745+0000 7f730e1ad740 1 -- 172.21.15.111:0/4016951919 wait complete. 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 -- start start 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 Processor -- start 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315b720 0x560c0316e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c0316b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73041add00 con 0x560c0315ae00 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f73041af170 con 0x560c03165fe0 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f73041ae3d0 con 0x560c0315b720 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.746+0000 7f730e1ad740 10 client.0 _ll_register_callbacks cb 0x7f72ec00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37822/0 (socket says 172.21.15.111:37822) 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 -- 172.21.15.111:0/1998637581 learned_addr learned my addr 172.21.15.111:0/1998637581 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f72fbe00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315b720 0x560c0316e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7309200640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c0316b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 -- 172.21.15.111:0/1998637581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315b720 msgr2=0x560c0316e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315b720 0x560c0316e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 -- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 msgr2=0x560c0316b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c0316b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 -- 172.21.15.111:0/1998637581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0316b8b0 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7309200640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c0316b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f7303e00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72f40057c0 tx=0x7f72f4030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51ee1290829930 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f72f2000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.747+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f405f070 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f72f40307f0 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f4057070 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72f4037050 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 10 client.15918 fetch_fsmap learned FSMap version 24 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560c031dd0e0 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4057070 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72f4062020 con 0x560c0315ae00 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 10 client.15918 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:50:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 20 client.15918 populate_metadata read hostname 'smithi111' 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c031dd850 0x560c031dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 4 client.15918 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.748+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c03157160 con 0x560c031dd850 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.749+0000 7f72fbe00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c031dd850 0x560c031dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.749+0000 7f72fbe00640 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c031dd850 0x560c031dfc70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3974e46f6ce4ea4 server_cookie=8e32f1b8ac6da63c in_seq=0 out_seq=0 2024-04-24T22:50:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.749+0000 7f72f2000640 10 client.15918 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f72f2000640 1 -- 172.21.15.111:0/1998637581 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c03157160 con 0x560c031dd850 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f72f2000640 10 client.15918 handle_command_reply: tid=0 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 client.15918 shutdown 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 msgr2=0x560c0316dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72f40057c0 tx=0x7f72f4030780 comp rx=0 tx=0).stop 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 shutdown_connections 2024-04-24T22:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x560c031dd850 0x560c031dfc70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c03165fe0 0x560c0316b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0315b720 0x560c0316e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f72f2000640 0 client.15918 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 --2- 172.21.15.111:0/1998637581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c0315ae00 0x560c0316dfe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 >> 172.21.15.111:0/1998637581 conn(0x560c03157310 msgr2=0x560c03162ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 shutdown_connections 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.765+0000 7f730e1ad740 1 -- 172.21.15.111:0/1998637581 wait complete. 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730e1ad740 20 client.15918 trim_cache size 0 max 16384 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 msgr2=0x7f72e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 0x7f72e80a5170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f72f402a970 tx=0x7f72f40056d0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 msgr2=0x7f730419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 shutdown_connections 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f72e80a2cb0 0x7f72e80a5170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7304109950 0x7f730419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7304104300 0x7f730419a720 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 --2- 172.21.15.111:0/2176557673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7304103940 0x7f730419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.766+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 >> 172.21.15.111:0/2176557673 conn(0x7f73040ff410 msgr2=0x7f73040ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.767+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 shutdown_connections 2024-04-24T22:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:50:47.767+0000 7f730b200640 1 -- 172.21.15.111:0/2176557673 wait complete. 2024-04-24T22:50:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:48 smithi171 ceph-mon[32337]: pgmap v274: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:48 smithi186 ceph-mon[32753]: pgmap v274: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:48.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:48 smithi111 ceph-mon[27746]: pgmap v274: 97 pgs: 97 active+clean; 4.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-24T22:50:50.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:50 smithi171 ceph-mon[32337]: pgmap v275: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:50.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:50 smithi186 ceph-mon[32753]: pgmap v275: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:50 smithi111 ceph-mon[27746]: pgmap v275: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:52.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:52 smithi171 ceph-mon[32337]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:50:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:52 smithi186 ceph-mon[32753]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:50:52.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:52 smithi111 ceph-mon[27746]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-24T22:50:54.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:54 smithi171 ceph-mon[32337]: pgmap v277: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:54.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:54 smithi186 ceph-mon[32753]: pgmap v277: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:54.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:54 smithi111 ceph-mon[27746]: pgmap v277: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-04-24T22:50:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:56 smithi171 ceph-mon[32337]: pgmap v278: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:56.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:56 smithi186 ceph-mon[32753]: pgmap v278: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:56 smithi111 ceph-mon[27746]: pgmap v278: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:50:58 smithi171 ceph-mon[32337]: pgmap v279: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:58.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:50:58 smithi186 ceph-mon[32753]: pgmap v279: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:50:58.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:50:58 smithi111 ceph-mon[27746]: pgmap v279: 97 pgs: 97 active+clean; 5.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:00 smithi171 ceph-mon[32337]: pgmap v280: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:51:00.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:00 smithi186 ceph-mon[32753]: pgmap v280: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:51:00.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:00 smithi111 ceph-mon[27746]: pgmap v280: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:51:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 -- 172.21.15.111:0/2281781776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc109950 msgr2=0x7fcabc109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 --2- 172.21.15.111:0/2281781776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc109950 0x7fcabc109d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009930 tx=0x7fcaac02a420 comp rx=0 tx=0).stop 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 -- 172.21.15.111:0/2281781776 shutdown_connections 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 --2- 172.21.15.111:0/2281781776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc104300 0x7fcabc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 --2- 172.21.15.111:0/2281781776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc103940 0x7fcabc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 --2- 172.21.15.111:0/2281781776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc109950 0x7fcabc109d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 -- 172.21.15.111:0/2281781776 >> 172.21.15.111:0/2281781776 conn(0x7fcabc0ff470 msgr2=0x7fcabc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 -- 172.21.15.111:0/2281781776 shutdown_connections 2024-04-24T22:51:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.911+0000 7fcac3200640 1 -- 172.21.15.111:0/2281781776 wait complete. 2024-04-24T22:51:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 Processor -- start 2024-04-24T22:51:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 -- start start 2024-04-24T22:51:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc103940 0x7fcabc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 0x7fcabc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 0x7fcabc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113d60 con 0x7fcabc109950 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113be0 con 0x7fcabc103940 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.912+0000 7fcac3200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113ee0 con 0x7fcabc104300 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 0x7fcabc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 0x7fcabc19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54960/0 (socket says 172.21.15.111:54960) 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcac1200640 1 -- 172.21.15.111:0/1072889721 learned_addr learned my addr 172.21.15.111:0/1072889721 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 0x7fcabc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc103940 0x7fcabc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 -- 172.21.15.111:0/1072889721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc103940 msgr2=0x7fcabc19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc103940 0x7fcabc19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 -- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 msgr2=0x7fcabc19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 0x7fcabc19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 -- 172.21.15.111:0/1072889721 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19f1c0 con 0x7fcabc104300 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.913+0000 7fcabb400640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 0x7fcabc19a760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9f0 tx=0x7fcaa400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.914+0000 7fcab8c00640 1 -- 172.21.15.111:0/1072889721 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400ccc0 con 0x7fcabc104300 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.914+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19f4b0 con 0x7fcabc104300 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.914+0000 7fcab8c00640 1 -- 172.21.15.111:0/1072889721 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa4010910 con 0x7fcabc104300 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.914+0000 7fcab8c00640 1 -- 172.21.15.111:0/1072889721 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4018960 con 0x7fcabc104300 2024-04-24T22:51:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.914+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a6d40 con 0x7fcabc104300 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 Processor -- start 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 -- start start 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcab8c00640 1 -- 172.21.15.111:0/1072889721 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcaa4010ac0 con 0x7fcabc104300 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 0x55aadec0a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec14aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec14fe0 0x55aadec173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aadeb6dab0 con 0x55aadec09e00 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aadeb16640 con 0x55aadec0a720 2024-04-24T22:51:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.917+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55aadeb69a70 con 0x55aadec14fe0 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 0x55aadec0a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabb400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec14fe0 0x55aadec173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec14aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 0x55aadec0a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54966/0 (socket says 172.21.15.111:54966) 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcac1200640 1 -- 172.21.15.111:0/1090682855 learned_addr learned my addr 172.21.15.111:0/1090682855 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 -- 172.21.15.111:0/1090682855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec14fe0 msgr2=0x55aadec173d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec14fe0 0x55aadec173d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 -- 172.21.15.111:0/1090682855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 msgr2=0x55aadec0a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 0x55aadec0a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.918+0000 7fcabbe00640 1 -- 172.21.15.111:0/1090682855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aade882210 con 0x55aadec0a720 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec14aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005940 tx=0x7fcaac005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba8bf22589852af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcab1600640 1 -- 172.21.15.111:0/1090682855 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac040070 con 0x55aadec0a720 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcab1600640 1 -- 172.21.15.111:0/1090682855 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac0317b0 con 0x55aadec0a720 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcab1600640 1 -- 172.21.15.111:0/1090682855 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac035040 con 0x55aadec0a720 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcab8c00640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 0x7fcaa00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcab8c00640 1 -- 172.21.15.111:0/1072889721 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fcaa40cf820 con 0x7fcabc104300 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.919+0000 7fcac1200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 0x7fcaa00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1090682855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 msgr2=0x55aadec14aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec14aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005940 tx=0x7fcaac005bd0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1090682855 shutdown_connections 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec14fe0 0x55aadec173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec14aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1090682855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec09e00 0x55aadec0a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1090682855 >> 172.21.15.111:0/1090682855 conn(0x55aadec06310 msgr2=0x55aadec066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1090682855 shutdown_connections 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1090682855 wait complete. 2024-04-24T22:51:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac1200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 0x7fcaa00a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcab40126c0 tx=0x7fcab4009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.920+0000 7fcac5f6a740 1 -- start start 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 Processor -- start 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec14fe0 0x55aadec12960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aadec17bb0 con 0x55aadec0a720 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aadeb29990 con 0x55aadec14fe0 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55aadec17d30 con 0x55aadec09e00 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac5f6a740 10 client.0 _ll_register_callbacks cb 0x55aadeb2b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcabb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcabbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcabb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54978/0 (socket says 172.21.15.111:54978) 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcabb400640 1 -- 172.21.15.111:0/1140833037 learned_addr learned my addr 172.21.15.111:0/1140833037 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.921+0000 7fcac1200640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec14fe0 0x55aadec12960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 -- 172.21.15.111:0/1140833037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 msgr2=0x55aadec11ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec11ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 -- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec14fe0 msgr2=0x55aadec12960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec14fe0 0x55aadec12960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 -- 172.21.15.111:0/1140833037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aade94e620 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcabb400640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcaa409ba90 tx=0x7fcaa40980f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92ec080d40927f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcaaa000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40cd3a0 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fcaa409b1e0 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac0317b0 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcaa409bd90 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcac5f6a740 10 client.15930 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.922+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aadec23a30 con 0x55aadec0a720 2024-04-24T22:51:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac0317b0 con 0x55aadec0a720 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcaa40b3050 con 0x55aadec0a720 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 20 client.15930 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec24ea0 0x55aadec272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 4 client.15930 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aadeb7aff0 con 0x55aadec24ea0 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.923+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec24ea0 0x55aadec272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.924+0000 7fcabbe00640 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec24ea0 0x55aadec272d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d394f171609e48d4 server_cookie=7c6f7a8183eb75d1 in_seq=0 out_seq=0 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.924+0000 7fcaaa000640 10 client.15930 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcaaa000640 1 -- 172.21.15.111:0/1140833037 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aadeb7aff0 con 0x55aadec24ea0 2024-04-24T22:51:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcaaa000640 10 client.15930 handle_command_reply: tid=0 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 client.15930 shutdown 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 msgr2=0x55aadec12420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcaa409ba90 tx=0x7fcaa40980f0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 shutdown_connections 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec24ea0 0x55aadec272d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec14fe0 0x55aadec12960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec0a720 0x55aadec12420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcaaa000640 0 client.15930 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/1140833037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec11ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 >> 172.21.15.111:0/1140833037 conn(0x55aadec06310 msgr2=0x55aadec0c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 shutdown_connections 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 1 -- 172.21.15.111:0/1140833037 wait complete. 2024-04-24T22:51:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.925+0000 7fcac5f6a740 20 client.15930 trim_cache size 0 max 16384 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.941+0000 7fcac5f6a740 1 Processor -- start 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.941+0000 7fcac5f6a740 1 -- start start 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec20a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec2cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1b1eb0 con 0x55aadec14fe0 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1af780 con 0x55aadec0a720 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ad5f0 con 0x55aadec09e00 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54994/0 (socket says 172.21.15.111:54994) 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcac1200640 1 -- 172.21.15.111:0/2489026827 learned_addr learned my addr 172.21.15.111:0/2489026827 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcabbe00640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec20a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.942+0000 7fcabb400640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec2cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 -- 172.21.15.111:0/2489026827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 msgr2=0x55aadec20a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec20a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 -- 172.21.15.111:0/2489026827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 msgr2=0x55aadec2d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 -- 172.21.15.111:0/2489026827 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aadec04890 con 0x55aadec0a720 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcac1200640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcabb400640 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec2cbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40981f0 tx=0x7fcaa4098160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aab3b8dd4f93e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcab0c00640 1 -- 172.21.15.111:0/2489026827 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40cd3a0 con 0x55aadec0a720 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcab0c00640 1 -- 172.21.15.111:0/2489026827 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa40a4950 con 0x55aadec0a720 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.943+0000 7fcab0c00640 1 -- 172.21.15.111:0/2489026827 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa409b4d0 con 0x55aadec0a720 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- 172.21.15.111:0/2489026827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 msgr2=0x55aadec2cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec2cbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40981f0 tx=0x7fcaa4098160 comp rx=0 tx=0).stop 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- 172.21.15.111:0/2489026827 shutdown_connections 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec2d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec0a720 0x55aadec2cbb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/2489026827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec09e00 0x55aadec20a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- 172.21.15.111:0/2489026827 >> 172.21.15.111:0/2489026827 conn(0x55aadec06310 msgr2=0x55aadec00fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- 172.21.15.111:0/2489026827 shutdown_connections 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- 172.21.15.111:0/2489026827 wait complete. 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 -- start start 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.944+0000 7fcac5f6a740 1 Processor -- start 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec09e00 0x55aadec18cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec92c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1adc40 con 0x55aadec14fe0 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1af0b0 con 0x55aadec09e00 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ae310 con 0x55aadec0a720 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac5f6a740 10 client.0 _ll_register_callbacks cb 0x7fcaa40d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcabb400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcac1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec92c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcabb400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:55606/0 (socket says 172.21.15.111:55606) 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcabb400640 1 -- 172.21.15.111:0/501086107 learned_addr learned my addr 172.21.15.111:0/501086107 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.945+0000 7fcabbe00640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec09e00 0x55aadec18cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 -- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec09e00 msgr2=0x55aadec18cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec09e00 0x55aadec18cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 -- 172.21.15.111:0/501086107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 msgr2=0x55aadec92c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec92c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 -- 172.21.15.111:0/501086107 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aadec19b00 con 0x55aadec0a720 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcac1200640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcabb400640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c6a0 tx=0x7fcaa400ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27abc2f65c6ae919 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcaaa000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40b9060 con 0x55aadec0a720 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fcaa4010640 con 0x55aadec0a720 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa40a4950 con 0x55aadec0a720 2024-04-24T22:51:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcaa40cd3a0 con 0x55aadec0a720 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcac5f6a740 10 client.25993 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.946+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aadec94280 con 0x55aadec0a720 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa40a4950 con 0x55aadec0a720 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcaa40b3030 con 0x55aadec0a720 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 10 client.25993 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 20 client.25993 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec94690 0x55aadec96ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 4 client.25993 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:51:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aadeb75340 con 0x55aadec94690 2024-04-24T22:51:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.947+0000 7fcabbe00640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec94690 0x55aadec96ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.948+0000 7fcabbe00640 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec94690 0x55aadec96ab0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8580a15b6a1fa1 server_cookie=9331b52cfc16c889 in_seq=0 out_seq=0 2024-04-24T22:51:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.948+0000 7fcaaa000640 10 client.25993 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcaaa000640 1 -- 172.21.15.111:0/501086107 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aadeb75340 con 0x55aadec94690 2024-04-24T22:51:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcaaa000640 10 client.25993 handle_command_reply: tid=0 2024-04-24T22:51:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 client.25993 shutdown 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 msgr2=0x55aadec19230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c6a0 tx=0x7fcaa400ced0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 shutdown_connections 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55aadec94690 0x55aadec96ab0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aadec14fe0 0x55aadec92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aadec0a720 0x55aadec19230 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 --2- 172.21.15.111:0/501086107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aadec09e00 0x55aadec18cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcaaa000640 0 client.25993 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 >> 172.21.15.111:0/501086107 conn(0x55aadec06310 msgr2=0x55aadec11ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 shutdown_connections 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 1 -- 172.21.15.111:0/501086107 wait complete. 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.962+0000 7fcac5f6a740 20 client.25993 trim_cache size 0 max 16384 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:51:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:51:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 msgr2=0x7fcaa00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 0x7fcaa00a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcab40126c0 tx=0x7fcab4009500 comp rx=0 tx=0).stop 2024-04-24T22:51:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 msgr2=0x7fcabc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 0x7fcabc19a760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9f0 tx=0x7fcaa400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 shutdown_connections 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc109950 0x7fcabc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcaa00a2da0 0x7fcaa00a5260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcabc104300 0x7fcabc19a760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 --2- 172.21.15.111:0/1072889721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcabc103940 0x7fcabc19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.963+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 >> 172.21.15.111:0/1072889721 conn(0x7fcabc0ff470 msgr2=0x7fcabc0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.964+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 shutdown_connections 2024-04-24T22:51:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:00.964+0000 7fcac3200640 1 -- 172.21.15.111:0/1072889721 wait complete. 2024-04-24T22:51:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:02 smithi171 ceph-mon[32337]: pgmap v281: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-24T22:51:02.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:02 smithi186 ceph-mon[32753]: pgmap v281: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-24T22:51:02.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:02 smithi111 ceph-mon[27746]: pgmap v281: 97 pgs: 97 active+clean; 5.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-24T22:51:04.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:04 smithi171 ceph-mon[32337]: pgmap v282: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 113 KiB/s wr, 38 op/s 2024-04-24T22:51:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:04 smithi186 ceph-mon[32753]: pgmap v282: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 113 KiB/s wr, 38 op/s 2024-04-24T22:51:04.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:04 smithi111 ceph-mon[27746]: pgmap v282: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 113 KiB/s wr, 38 op/s 2024-04-24T22:51:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:06 smithi171 ceph-mon[32337]: pgmap v283: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 30 op/s 2024-04-24T22:51:06.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:06 smithi186 ceph-mon[32753]: pgmap v283: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 30 op/s 2024-04-24T22:51:06.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:06 smithi111 ceph-mon[27746]: pgmap v283: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 30 op/s 2024-04-24T22:51:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:07 smithi186 ceph-mon[32753]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T22:51:07.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:07 smithi111 ceph-mon[27746]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T22:51:07.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:07 smithi171 ceph-mon[32337]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T22:51:08.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:08 smithi186 ceph-mon[32753]: pgmap v285: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 36 op/s 2024-04-24T22:51:08.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:08 smithi111 ceph-mon[27746]: pgmap v285: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 36 op/s 2024-04-24T22:51:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:08 smithi171 ceph-mon[32337]: pgmap v285: 97 pgs: 97 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 36 op/s 2024-04-24T22:51:10.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:10 smithi186 ceph-mon[32753]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:10.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:10 smithi111 ceph-mon[27746]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:10 smithi171 ceph-mon[32337]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:11.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:11 smithi186 ceph-mon[32753]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:11.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:11 smithi111 ceph-mon[27746]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:11.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:11 smithi171 ceph-mon[32337]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-04-24T22:51:13.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:13 smithi186 ceph-mon[32753]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T22:51:13.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:13 smithi111 ceph-mon[27746]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T22:51:13.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:13 smithi171 ceph-mon[32337]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 -- 172.21.15.111:0/3870347648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b4076590 msgr2=0x7ff4b41120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 --2- 172.21.15.111:0/3870347648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b4076590 0x7ff4b41120a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009860 tx=0x7ff4a002a340 comp rx=0 tx=0).stop 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 -- 172.21.15.111:0/3870347648 shutdown_connections 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 --2- 172.21.15.111:0/3870347648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4b41125e0 0x7ff4b41149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 --2- 172.21.15.111:0/3870347648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b4076590 0x7ff4b41120a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 --2- 172.21.15.111:0/3870347648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b4075c70 0x7ff4b4076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 -- 172.21.15.111:0/3870347648 >> 172.21.15.111:0/3870347648 conn(0x7ff4b40ff470 msgr2=0x7ff4b4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 -- 172.21.15.111:0/3870347648 shutdown_connections 2024-04-24T22:51:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.108+0000 7ff4bc400640 1 -- 172.21.15.111:0/3870347648 wait complete. 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.109+0000 7ff4bc400640 1 Processor -- start 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.109+0000 7ff4bc400640 1 -- start start 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b4075c70 0x7ff4b419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4b4076590 0x7ff4b419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4118140 con 0x7ff4b41125e0 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4117fc0 con 0x7ff4b4075c70 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4bc400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41182c0 con 0x7ff4b4076590 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54986/0 (socket says 172.21.15.111:54986) 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 -- 172.21.15.111:0/86322656 learned_addr learned my addr 172.21.15.111:0/86322656 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 -- 172.21.15.111:0/86322656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4b4076590 msgr2=0x7ff4b419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4b4076590 0x7ff4b419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 -- 172.21.15.111:0/86322656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b4075c70 msgr2=0x7ff4b419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b4075c70 0x7ff4b419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.110+0000 7ff4ba400640 1 -- 172.21.15.111:0/86322656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41a35e0 con 0x7ff4b41125e0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4ba400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fa0 tx=0x7ff4ac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4b2000640 1 -- 172.21.15.111:0/86322656 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac004b20 con 0x7ff4b41125e0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4b2000640 1 -- 172.21.15.111:0/86322656 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac003e50 con 0x7ff4b41125e0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4b2000640 1 -- 172.21.15.111:0/86322656 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00b570 con 0x7ff4b41125e0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a38d0 con 0x7ff4b41125e0 2024-04-24T22:51:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.111+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41ab160 con 0x7ff4b41125e0 2024-04-24T22:51:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.114+0000 7ff4bf1a2740 1 Processor -- start 2024-04-24T22:51:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.114+0000 7ff4b2000640 1 -- 172.21.15.111:0/86322656 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff4ac01d020 con 0x7ff4b41125e0 2024-04-24T22:51:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 -- start start 2024-04-24T22:51:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d55aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558ae0caeab0 con 0x558ae0d4ae00 2024-04-24T22:51:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558ae0c57640 con 0x558ae0d4b720 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558ae0caaa70 con 0x558ae0d55fe0 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54994/0 (socket says 172.21.15.111:54994) 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4b9000640 1 -- 172.21.15.111:0/2194559805 learned_addr learned my addr 172.21.15.111:0/2194559805 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.115+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d55aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 -- 172.21.15.111:0/2194559805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 msgr2=0x558ae0d583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 -- 172.21.15.111:0/2194559805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 msgr2=0x558ae0d55aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d55aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 -- 172.21.15.111:0/2194559805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae09c3210 con 0x558ae0d4ae00 2024-04-24T22:51:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b2000640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 0x7ff49c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00057c0 tx=0x7ff4a0005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f755f5867b809c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b2000640 1 -- 172.21.15.111:0/86322656 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7ff4ac0cf470 con 0x7ff4b41125e0 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 0x7ff49c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.116+0000 7ff4a6a00640 1 -- 172.21.15.111:0/2194559805 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037070 con 0x558ae0d4ae00 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4a6a00640 1 -- 172.21.15.111:0/2194559805 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0004040 con 0x558ae0d4ae00 2024-04-24T22:51:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4a6a00640 1 -- 172.21.15.111:0/2194559805 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00054c0 con 0x558ae0d4ae00 2024-04-24T22:51:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2194559805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 msgr2=0x558ae0d4b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 0x7ff49c0a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80095f0 tx=0x7ff4a80094a0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00057c0 tx=0x7ff4a0005b20 comp rx=0 tx=0).stop 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2194559805 shutdown_connections 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d55aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2194559805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4ae00 0x558ae0d4b1e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2194559805 >> 172.21.15.111:0/2194559805 conn(0x558ae0d47310 msgr2=0x558ae0d476f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.117+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2194559805 shutdown_connections 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.118+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2194559805 wait complete. 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.118+0000 7ff4bf1a2740 1 -- start start 2024-04-24T22:51:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.118+0000 7ff4bf1a2740 1 Processor -- start 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d53490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558ae0d58b50 con 0x558ae0d4b720 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558ae0c6a990 con 0x558ae0d4ae00 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558ae0d58cd0 con 0x558ae0d55fe0 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4bf1a2740 10 client.0 _ll_register_callbacks cb 0x558ae0c6c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:37932/0 (socket says 172.21.15.111:37932) 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 -- 172.21.15.111:0/2671592142 learned_addr learned my addr 172.21.15.111:0/2671592142 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d53490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4ba400640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 -- 172.21.15.111:0/2671592142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 msgr2=0x558ae0d539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 -- 172.21.15.111:0/2671592142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 msgr2=0x558ae0d53490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d53490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.119+0000 7ff4b9a00640 1 -- 172.21.15.111:0/2671592142 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae0a8f620 con 0x558ae0d4ae00 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801c9c0 tx=0x7ff4a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c03834d6534fb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff49b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800eab0 con 0x558ae0d4ae00 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7ff4a800ec50 con 0x558ae0d4ae00 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a0004040 con 0x558ae0d4ae00 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff4a800ee50 con 0x558ae0d4ae00 2024-04-24T22:51:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff4bf1a2740 10 client.25961 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.120+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558ae0d64af0 con 0x558ae0d4ae00 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0004040 con 0x558ae0d4ae00 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff4a801a550 con 0x558ae0d4ae00 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 10 client.25961 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 20 client.25961 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0d66000 0x558ae0d68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 4 client.25961 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae0cbbff0 con 0x558ae0d66000 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.121+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0d66000 0x558ae0d68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.122+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0d66000 0x558ae0d68410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fe3d7bd2f6a26 server_cookie=cfc32bd06dd9c15f in_seq=0 out_seq=0 2024-04-24T22:51:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.122+0000 7ff49b400640 10 client.25961 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff49b400640 1 -- 172.21.15.111:0/2671592142 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ae0cbbff0 con 0x558ae0d66000 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff49b400640 10 client.25961 handle_command_reply: tid=0 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 client.25961 shutdown 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 msgr2=0x558ae0d52f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801c9c0 tx=0x7ff4a80046c0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 shutdown_connections 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0d66000 0x558ae0d68410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d55fe0 0x558ae0d539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d53490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff49b400640 0 client.25961 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2671592142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4ae00 0x558ae0d52f50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 >> 172.21.15.111:0/2671592142 conn(0x558ae0d47310 msgr2=0x558ae0d4d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 shutdown_connections 2024-04-24T22:51:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2671592142 wait complete. 2024-04-24T22:51:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.123+0000 7ff4bf1a2740 20 client.25961 trim_cache size 0 max 16384 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 Processor -- start 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 -- start start 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d55fe0 0x558ae0d537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b6220 con 0x558ae0d4b720 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b3af0 con 0x558ae0d55fe0 2024-04-24T22:51:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.140+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b1960 con 0x558ae0d4ae00 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55014/0 (socket says 172.21.15.111:55014) 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 -- 172.21.15.111:0/1542791598 learned_addr learned my addr 172.21.15.111:0/1542791598 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4ba400640 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d55fe0 0x558ae0d537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 -- 172.21.15.111:0/1542791598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 msgr2=0x558ae0d4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d4d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 -- 172.21.15.111:0/1542791598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d55fe0 msgr2=0x558ae0d537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d55fe0 0x558ae0d537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 -- 172.21.15.111:0/1542791598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae0d4b550 con 0x558ae0d4b720 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4b9000640 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002a860 tx=0x7ff4a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b7b6d0a7ab3a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4a6000640 1 -- 172.21.15.111:0/1542791598 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037040 con 0x558ae0d4b720 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.141+0000 7ff4a6000640 1 -- 172.21.15.111:0/1542791598 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0005850 con 0x558ae0d4b720 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4a6000640 1 -- 172.21.15.111:0/1542791598 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00046c0 con 0x558ae0d4b720 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/1542791598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 msgr2=0x558ae0d46300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002a860 tx=0x7ff4a0005f90 comp rx=0 tx=0).stop 2024-04-24T22:51:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/1542791598 shutdown_connections 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d55fe0 0x558ae0d537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d4b720 0x558ae0d46300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/1542791598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/1542791598 >> 172.21.15.111:0/1542791598 conn(0x558ae0d47310 msgr2=0x558ae0d45bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/1542791598 shutdown_connections 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.142+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/1542791598 wait complete. 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 -- start start 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 Processor -- start 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d59980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d59ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b1fb0 con 0x558ae0d55fe0 2024-04-24T22:51:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b3420 con 0x558ae0d4b720 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b2680 con 0x558ae0d4ae00 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.143+0000 7ff4bf1a2740 10 client.0 _ll_register_callbacks cb 0x7ff4a80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55018/0 (socket says 172.21.15.111:55018) 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 -- 172.21.15.111:0/2523125015 learned_addr learned my addr 172.21.15.111:0/2523125015 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d59980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4b9000640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d59ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 -- 172.21.15.111:0/2523125015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 msgr2=0x558ae0d59980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d59980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 -- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 msgr2=0x558ae0d59ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d59ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 -- 172.21.15.111:0/2523125015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae0dcb5e0 con 0x558ae0d55fe0 2024-04-24T22:51:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff4ba400640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac004330 tx=0x7ff4ac09b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe99b18e3e4d15fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff49b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.144+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01a040 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7ff4ac097fa0 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a0005850 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff4ac09c030 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 10 client.15954 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558ae0dcc930 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0005850 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff4ac00b040 con 0x558ae0d55fe0 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 10 client.15954 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 20 client.15954 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0dcd060 0x558ae0dcf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 4 client.15954 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.145+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ae0cb6340 con 0x558ae0dcd060 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.146+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0dcd060 0x558ae0dcf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.146+0000 7ff4b9a00640 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0dcd060 0x558ae0dcf480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828545b0ed31c7f1 server_cookie=c07e9c6beea78f83 in_seq=0 out_seq=0 2024-04-24T22:51:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.146+0000 7ff49b400640 10 client.15954 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff49b400640 1 -- 172.21.15.111:0/2523125015 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ae0cb6340 con 0x558ae0dcd060 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff49b400640 10 client.15954 handle_command_reply: tid=0 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 client.15954 shutdown 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 msgr2=0x558ae0d5a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac004330 tx=0x7ff4ac09b470 comp rx=0 tx=0).stop 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 shutdown_connections 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x558ae0dcd060 0x558ae0dcf480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ae0d55fe0 0x558ae0d5a400 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ae0d4b720 0x558ae0d59ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff49b400640 0 client.15954 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 --2- 172.21.15.111:0/2523125015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ae0d4ae00 0x558ae0d59980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 >> 172.21.15.111:0/2523125015 conn(0x558ae0d47310 msgr2=0x558ae0d46480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 shutdown_connections 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.160+0000 7ff4bf1a2740 1 -- 172.21.15.111:0/2523125015 wait complete. 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.161+0000 7ff4bf1a2740 20 client.15954 trim_cache size 0 max 16384 2024-04-24T22:51:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:51:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:51:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:51:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:51:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.161+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 msgr2=0x7ff49c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.161+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 0x7ff49c0a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80095f0 tx=0x7ff4a80094a0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.161+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 msgr2=0x7ff4b41a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.161+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fa0 tx=0x7ff4ac00c800 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 shutdown_connections 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4b41125e0 0x7ff4b41a2eb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7ff49c0a2e20 0x7ff49c0a52e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4b4076590 0x7ff4b419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 --2- 172.21.15.111:0/86322656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b4075c70 0x7ff4b419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 >> 172.21.15.111:0/86322656 conn(0x7ff4b40ff470 msgr2=0x7ff4b4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 shutdown_connections 2024-04-24T22:51:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:14.162+0000 7ff4bc400640 1 -- 172.21.15.111:0/86322656 wait complete. 2024-04-24T22:51:14.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:14 smithi186 ceph-mon[32753]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:51:14.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:14 smithi111 ceph-mon[27746]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:51:14.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:14 smithi171 ceph-mon[32337]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-24T22:51:16.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:16 smithi186 ceph-mon[32753]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:51:16.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:16 smithi111 ceph-mon[27746]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:51:16.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:16 smithi171 ceph-mon[32337]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:51:18.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:18 smithi186 ceph-mon[32753]: pgmap v291: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:51:18.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:18 smithi111 ceph-mon[27746]: pgmap v291: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:51:18.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:18 smithi171 ceph-mon[32337]: pgmap v291: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2024-04-24T22:51:19.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:19 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:19.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:19 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:19.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:19 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:20 smithi186 ceph-mon[32753]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:20 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:20 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:20.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:20 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:51:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:20 smithi111 ceph-mon[27746]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:20 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:20 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:20.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:20 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:51:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:20 smithi171 ceph-mon[32337]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:20 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:20 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:20 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:51:22.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:22 smithi186 ceph-mon[32753]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:22.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:22 smithi111 ceph-mon[27746]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:22 smithi171 ceph-mon[32337]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-24T22:51:24.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:24 smithi186 ceph-mon[32753]: pgmap v294: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 72 KiB/s wr, 20 op/s 2024-04-24T22:51:24.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:24 smithi111 ceph-mon[27746]: pgmap v294: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 72 KiB/s wr, 20 op/s 2024-04-24T22:51:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:24 smithi171 ceph-mon[32337]: pgmap v294: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 72 KiB/s wr, 20 op/s 2024-04-24T22:51:26.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:26 smithi186 ceph-mon[32753]: pgmap v295: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:26.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:26 smithi111 ceph-mon[27746]: pgmap v295: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:26 smithi171 ceph-mon[32337]: pgmap v295: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 -- 172.21.15.111:0/4288158382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a701125e0 msgr2=0x7f8a701149d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 --2- 172.21.15.111:0/4288158382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a701125e0 0x7f8a701149d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009cd0 tx=0x7f8a6802a8f0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 -- 172.21.15.111:0/4288158382 shutdown_connections 2024-04-24T22:51:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 --2- 172.21.15.111:0/4288158382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a701125e0 0x7f8a701149d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 --2- 172.21.15.111:0/4288158382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a70076590 0x7f8a701120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 --2- 172.21.15.111:0/4288158382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70075c70 0x7f8a70076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 -- 172.21.15.111:0/4288158382 >> 172.21.15.111:0/4288158382 conn(0x7f8a700ff470 msgr2=0x7f8a70101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 -- 172.21.15.111:0/4288158382 shutdown_connections 2024-04-24T22:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.303+0000 7f8a78200640 1 -- 172.21.15.111:0/4288158382 wait complete. 2024-04-24T22:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.304+0000 7f8a78200640 1 Processor -- start 2024-04-24T22:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 -- start start 2024-04-24T22:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a70075c70 0x7f8a7019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 0x7f8a7019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70118150 con 0x7f8a701125e0 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70117fd0 con 0x7f8a70076590 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a78200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701182d0 con 0x7f8a70075c70 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a76200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a76200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59496/0 (socket says 172.21.15.111:59496) 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a76200640 1 -- 172.21.15.111:0/1165619396 learned_addr learned my addr 172.21.15.111:0/1165619396 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.305+0000 7f8a74e00640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 0x7f8a7019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a75800640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a70075c70 0x7f8a7019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 -- 172.21.15.111:0/1165619396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a70075c70 msgr2=0x7f8a7019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a70075c70 0x7f8a7019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 -- 172.21.15.111:0/1165619396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 msgr2=0x7f8a7019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 0x7f8a7019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 -- 172.21.15.111:0/1165619396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a701a3550 con 0x7f8a701125e0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a74e00640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 0x7f8a7019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a76200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005e30 tx=0x7f8a68003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.306+0000 7f8a6e000640 1 -- 172.21.15.111:0/1165619396 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68030dc0 con 0x7f8a701125e0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.307+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a701a37e0 con 0x7f8a701125e0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.307+0000 7f8a6e000640 1 -- 172.21.15.111:0/1165619396 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a68040070 con 0x7f8a701125e0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.307+0000 7f8a6e000640 1 -- 172.21.15.111:0/1165619396 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68045aa0 con 0x7f8a701125e0 2024-04-24T22:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.307+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a701ab0d0 con 0x7f8a701125e0 2024-04-24T22:51:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.310+0000 7f8a6e000640 1 -- 172.21.15.111:0/1165619396 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8a68004700 con 0x7f8a701125e0 2024-04-24T22:51:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.310+0000 7f8a7b079740 1 Processor -- start 2024-04-24T22:51:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.310+0000 7f8a7b079740 1 -- start start 2024-04-24T22:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.310+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e2e00 0x55a2203e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.310+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203edfe0 0x55a2203f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a7b079740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a220346ab0 con 0x55a2203edfe0 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a7b079740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a2202ef640 con 0x55a2203e2e00 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a7b079740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a220342a70 con 0x55a2203e3720 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53444/0 (socket says 172.21.15.111:53444) 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 -- 172.21.15.111:0/3765649710 learned_addr learned my addr 172.21.15.111:0/3765649710 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a74e00640 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e2e00 0x55a2203e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a74e00640 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203edfe0 0x55a2203f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 -- 172.21.15.111:0/3765649710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e2e00 msgr2=0x55a2203e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e2e00 0x55a2203e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 -- 172.21.15.111:0/3765649710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203edfe0 msgr2=0x55a2203f03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203edfe0 0x55a2203f03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.311+0000 7f8a75800640 1 -- 172.21.15.111:0/3765649710 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22005b210 con 0x55a2203e3720 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a75800640 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009cf0 tx=0x7f8a64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b9a8f04b1dd54d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a6e000640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 0x7f8a580a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a5ea00640 1 -- 172.21.15.111:0/3765649710 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64004230 con 0x55a2203e3720 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a5ea00640 1 -- 172.21.15.111:0/3765649710 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a640043d0 con 0x55a2203e3720 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a5ea00640 1 -- 172.21.15.111:0/3765649710 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64019960 con 0x55a2203e3720 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a6e000640 1 -- 172.21.15.111:0/1165619396 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8a680f02d0 con 0x7f8a701125e0 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.312+0000 7f8a74e00640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 0x7f8a580a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- 172.21.15.111:0/3765649710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 msgr2=0x55a2203edaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a74e00640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 0x7f8a580a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55a2203e0730 tx=0x7f8a60013040 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009cf0 tx=0x7f8a64011810 comp rx=0 tx=0).stop 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- 172.21.15.111:0/3765649710 shutdown_connections 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203edfe0 0x55a2203f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e3720 0x55a2203edaa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 --2- 172.21.15.111:0/3765649710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e2e00 0x55a2203e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- 172.21.15.111:0/3765649710 >> 172.21.15.111:0/3765649710 conn(0x55a2203df310 msgr2=0x55a2203df6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- 172.21.15.111:0/3765649710 shutdown_connections 2024-04-24T22:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- 172.21.15.111:0/3765649710 wait complete. 2024-04-24T22:51:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.313+0000 7f8a7b079740 1 -- start start 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 Processor -- start 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203eba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a2203f0c10 con 0x55a2203e2e00 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a220302990 con 0x55a2203e3720 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2203f0d90 con 0x55a2203edfe0 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a7b079740 10 client.0 _ll_register_callbacks cb 0x55a220304950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a75800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a75800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59520/0 (socket says 172.21.15.111:59520) 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.314+0000 7f8a75800640 1 -- 172.21.15.111:0/2858128025 learned_addr learned my addr 172.21.15.111:0/2858128025 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a76200640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 -- 172.21.15.111:0/2858128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 msgr2=0x55a2203eba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203eba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 -- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 msgr2=0x55a2203eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 -- 172.21.15.111:0/2858128025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a220127620 con 0x55a2203e2e00 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a75800640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8a64003e00 tx=0x7f8a64003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e80ab1c28c16cae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a53400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.315+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64027070 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8a64005240 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a640043d0 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8a64033050 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 10 client.15966 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a2203fcb40 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a640043d0 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8a64004ba0 con 0x55a2203e2e00 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 10 client.15966 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.316+0000 7f8a7b079740 20 client.15966 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.317+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a2203fdf50 0x55a220400340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.317+0000 7f8a7b079740 4 client.15966 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:51:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.317+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a220353ff0 con 0x55a2203fdf50 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.317+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a2203fdf50 0x55a220400340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.318+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a2203fdf50 0x55a220400340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337cbe27b75e27ad server_cookie=524cdfa3a0aaaac7 in_seq=0 out_seq=0 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.318+0000 7f8a53400640 10 client.15966 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.318+0000 7f8a53400640 1 -- 172.21.15.111:0/2858128025 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a220353ff0 con 0x55a2203fdf50 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.318+0000 7f8a53400640 10 client.15966 handle_command_reply: tid=0 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 client.15966 shutdown 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 msgr2=0x55a2203eaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8a64003e00 tx=0x7f8a64003820 comp rx=0 tx=0).stop 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 shutdown_connections 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a2203fdf50 0x55a220400340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203eba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a53400640 0 client.15966 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2858128025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203eaf80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 >> 172.21.15.111:0/2858128025 conn(0x55a2203df310 msgr2=0x55a2203e5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 shutdown_connections 2024-04-24T22:51:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 1 -- 172.21.15.111:0/2858128025 wait complete. 2024-04-24T22:51:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.319+0000 7f8a7b079740 20 client.15966 trim_cache size 0 max 16384 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 Processor -- start 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 -- start start 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e2e00 0x55a2203f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203edfe0 0x55a2203eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b61b0 con 0x55a2203e3720 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b3a80 con 0x55a2203edfe0 2024-04-24T22:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a7b079740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b18f0 con 0x55a2203e2e00 2024-04-24T22:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a74e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a75800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e2e00 0x55a2203f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a74e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59526/0 (socket says 172.21.15.111:59526) 2024-04-24T22:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.336+0000 7f8a74e00640 1 -- 172.21.15.111:0/2272125250 learned_addr learned my addr 172.21.15.111:0/2272125250 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 -- 172.21.15.111:0/2272125250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e2e00 msgr2=0x55a2203f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e2e00 0x55a2203f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 -- 172.21.15.111:0/2272125250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203edfe0 msgr2=0x55a2203eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203edfe0 0x55a2203eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 -- 172.21.15.111:0/2272125250 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2203da620 con 0x55a2203e3720 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009fd0 tx=0x7f8a600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c150e9f2493a021 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.337+0000 7f8a5e000640 1 -- 172.21.15.111:0/2272125250 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600040f0 con 0x55a2203e3720 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a5e000640 1 -- 172.21.15.111:0/2272125250 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a60021e20 con 0x55a2203e3720 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a5e000640 1 -- 172.21.15.111:0/2272125250 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600035f0 con 0x55a2203e3720 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 -- 172.21.15.111:0/2272125250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 msgr2=0x55a2203e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009fd0 tx=0x7f8a600046c0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 -- 172.21.15.111:0/2272125250 shutdown_connections 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203edfe0 0x55a2203eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e3720 0x55a2203e4ec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2272125250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203e2e00 0x55a2203f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 -- 172.21.15.111:0/2272125250 >> 172.21.15.111:0/2272125250 conn(0x55a2203df310 msgr2=0x55a2203e47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 -- 172.21.15.111:0/2272125250 shutdown_connections 2024-04-24T22:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.338+0000 7f8a7b079740 1 -- 172.21.15.111:0/2272125250 wait complete. 2024-04-24T22:51:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 -- start start 2024-04-24T22:51:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 Processor -- start 2024-04-24T22:51:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b1f40 con 0x55a2203e2e00 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b33b0 con 0x55a2203e3720 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.339+0000 7f8a7b079740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a701b2610 con 0x55a2203edfe0 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a7b079740 10 client.0 _ll_register_callbacks cb 0x7f8a60009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a75800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a76200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:59412/0 (socket says 172.21.15.111:59412) 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 -- 172.21.15.111:0/2753206916 learned_addr learned my addr 172.21.15.111:0/2753206916 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 -- 172.21.15.111:0/2753206916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 msgr2=0x55a2203e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 -- 172.21.15.111:0/2753206916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 msgr2=0x55a2203f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 -- 172.21.15.111:0/2753206916 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2203e8420 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a74e00640 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005fb0 tx=0x7f8a60009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e0a2cfce429e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.340+0000 7f8a53400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6002c3c0 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8a60009040 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a60021e20 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8a6002c560 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a7b079740 10 client.25982 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a220464220 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.341+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a60021e20 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8a60003a30 con 0x55a2203e3720 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 10 client.25982 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 20 client.25982 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a220464960 0x55a220466d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 4 client.25982 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a22034e340 con 0x55a220464960 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a75800640 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a220464960 0x55a220466d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a75800640 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a220464960 0x55a220466d80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2f752f2cb14b00 server_cookie=af6f2ec11b1307ee in_seq=0 out_seq=0 2024-04-24T22:51:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.342+0000 7f8a53400640 10 client.25982 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a53400640 1 -- 172.21.15.111:0/2753206916 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a22034e340 con 0x55a220464960 2024-04-24T22:51:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a53400640 10 client.25982 handle_command_reply: tid=0 2024-04-24T22:51:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 client.25982 shutdown 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 msgr2=0x55a2203e7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005fb0 tx=0x7f8a60009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 shutdown_connections 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a220464960 0x55a220466d80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2203edfe0 0x55a2203e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a2203e3720 0x55a2203e7870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 --2- 172.21.15.111:0/2753206916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a2203e2e00 0x55a2203f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a53400640 0 client.25982 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 >> 172.21.15.111:0/2753206916 conn(0x55a2203df310 msgr2=0x55a2203e5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 shutdown_connections 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.356+0000 7f8a7b079740 1 -- 172.21.15.111:0/2753206916 wait complete. 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a7b079740 20 client.25982 trim_cache size 0 max 16384 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:51:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:51:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 msgr2=0x7f8a580a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 0x7f8a580a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55a2203e0730 tx=0x7f8a60013040 comp rx=0 tx=0).stop 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 msgr2=0x7f8a701a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005e30 tx=0x7f8a68003f10 comp rx=0 tx=0).stop 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 shutdown_connections 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a701125e0 0x7f8a701a2e20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a70076590 0x7f8a7019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a580a2da0 0x7f8a580a5260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 --2- 172.21.15.111:0/1165619396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a70075c70 0x7f8a7019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.357+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 >> 172.21.15.111:0/1165619396 conn(0x7f8a700ff470 msgr2=0x7f8a70101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.358+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 shutdown_connections 2024-04-24T22:51:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:27.358+0000 7f8a78200640 1 -- 172.21.15.111:0/1165619396 wait complete. 2024-04-24T22:51:28.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:28 smithi186 ceph-mon[32753]: pgmap v296: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:28.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:28 smithi111 ceph-mon[27746]: pgmap v296: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:28 smithi171 ceph-mon[32337]: pgmap v296: 97 pgs: 97 active+clean; 2.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:51:30.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:30 smithi111 ceph-mon[27746]: pgmap v297: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:51:30.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:30 smithi171 ceph-mon[32337]: pgmap v297: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:51:30.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:30 smithi186 ceph-mon[32753]: pgmap v297: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:51:32.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:32 smithi111 ceph-mon[27746]: pgmap v298: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:32 smithi171 ceph-mon[32337]: pgmap v298: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:32 smithi186 ceph-mon[32753]: pgmap v298: 97 pgs: 97 active+clean; 2.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:34.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:34 smithi111 ceph-mon[27746]: pgmap v299: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:34 smithi171 ceph-mon[32337]: pgmap v299: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:34 smithi186 ceph-mon[32753]: pgmap v299: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:36.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:36 smithi111 ceph-mon[27746]: pgmap v300: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:36.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:36 smithi186 ceph-mon[32753]: pgmap v300: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:36.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:36 smithi171 ceph-mon[32337]: pgmap v300: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:38.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:38 smithi111 ceph-mon[27746]: pgmap v301: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:38 smithi171 ceph-mon[32337]: pgmap v301: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:38 smithi186 ceph-mon[32753]: pgmap v301: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 -- 172.21.15.111:0/4048337324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100df0 msgr2=0x7efdf010d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 --2- 172.21.15.111:0/4048337324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100df0 0x7efdf010d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efde8009cd0 tx=0x7efde802a8e0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 -- 172.21.15.111:0/4048337324 shutdown_connections 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 --2- 172.21.15.111:0/4048337324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100df0 0x7efdf010d300 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 --2- 172.21.15.111:0/4048337324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100430 0x7efdf01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 --2- 172.21.15.111:0/4048337324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf01068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.499+0000 7efdf4a00640 1 -- 172.21.15.111:0/4048337324 >> 172.21.15.111:0/4048337324 conn(0x7efdf00fbf60 msgr2=0x7efdf00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.500+0000 7efdf4a00640 1 -- 172.21.15.111:0/4048337324 shutdown_connections 2024-04-24T22:51:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.500+0000 7efdf4a00640 1 -- 172.21.15.111:0/4048337324 wait complete. 2024-04-24T22:51:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 Processor -- start 2024-04-24T22:51:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 -- start start 2024-04-24T22:51:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100df0 0x7efdf00fc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf00fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0110830 con 0x7efdf0100430 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01106b0 con 0x7efdf01064d0 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdf4a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01109b0 con 0x7efdf0100df0 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38662/0 (socket says 172.21.15.111:38662) 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.501+0000 7efdef400640 1 -- 172.21.15.111:0/1494171954 learned_addr learned my addr 172.21.15.111:0/1494171954 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdefe00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf00fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efde7e00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100df0 0x7efdf00fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdef400640 1 -- 172.21.15.111:0/1494171954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100df0 msgr2=0x7efdf00fc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdef400640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100df0 0x7efdf00fc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdef400640 1 -- 172.21.15.111:0/1494171954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 msgr2=0x7efdf00fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdef400640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf00fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdef400640 1 -- 172.21.15.111:0/1494171954 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf01a6310 con 0x7efdf0100430 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.502+0000 7efdefe00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf00fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdef400640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efde000e9f0 tx=0x7efde000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdecc00640 1 -- 172.21.15.111:0/1494171954 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000cd80 con 0x7efdf0100430 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdecc00640 1 -- 172.21.15.111:0/1494171954 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde0010910 con 0x7efdf0100430 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdecc00640 1 -- 172.21.15.111:0/1494171954 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0018990 con 0x7efdf0100430 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf01a6600 con 0x7efdf0100430 2024-04-24T22:51:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.503+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a6b00 con 0x7efdf0100430 2024-04-24T22:51:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.506+0000 7efdecc00640 1 -- 172.21.15.111:0/1494171954 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efde0010ac0 con 0x7efdf0100430 2024-04-24T22:51:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.506+0000 7efdf8e46740 1 Processor -- start 2024-04-24T22:51:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.506+0000 7efdf8e46740 1 -- start start 2024-04-24T22:51:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c282390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 0x55a06c28cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c28f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a06c1e5ab0 con 0x55a06c2828d0 2024-04-24T22:51:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a06c18e640 con 0x55a06c28d190 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdf8e46740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a06c1e1a70 con 0x55a06c281fb0 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 0x55a06c28cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdefe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c282390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efde7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c28f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 0x55a06c28cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38666/0 (socket says 172.21.15.111:38666) 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efdef400640 1 -- 172.21.15.111:0/3299884954 learned_addr learned my addr 172.21.15.111:0/3299884954 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efde7e00640 1 -- 172.21.15.111:0/3299884954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 msgr2=0x55a06c282390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.507+0000 7efde7e00640 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c282390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efde7e00640 1 -- 172.21.15.111:0/3299884954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 msgr2=0x55a06c28cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efde7e00640 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 0x55a06c28cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efde7e00640 1 -- 172.21.15.111:0/3299884954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06befa210 con 0x55a06c28d190 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efde7e00640 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c28f580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009bf0 tx=0x7efdd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1fda4e37edb2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efdecc00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 0x7efdd40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efddfe00640 1 -- 172.21.15.111:0/3299884954 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd801e070 con 0x55a06c28d190 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efddfe00640 1 -- 172.21.15.111:0/3299884954 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd8009da0 con 0x55a06c28d190 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efddfe00640 1 -- 172.21.15.111:0/3299884954 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8018870 con 0x55a06c28d190 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efdecc00640 1 -- 172.21.15.111:0/1494171954 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7efde00cf680 con 0x7efdf0100430 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.508+0000 7efdefe00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 0x7efdd40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 -- 172.21.15.111:0/3299884954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 msgr2=0x55a06c28f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c28f580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009bf0 tx=0x7efdd8011810 comp rx=0 tx=0).stop 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 -- 172.21.15.111:0/3299884954 shutdown_connections 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c28f580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c2828d0 0x55a06c28cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 --2- 172.21.15.111:0/3299884954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c282390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 -- 172.21.15.111:0/3299884954 >> 172.21.15.111:0/3299884954 conn(0x55a06c27e4c0 msgr2=0x55a06c27e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdefe00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 0x7efdd40a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efde8005e30 tx=0x7efde803d040 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 -- 172.21.15.111:0/3299884954 shutdown_connections 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.509+0000 7efdf8e46740 1 -- 172.21.15.111:0/3299884954 wait complete. 2024-04-24T22:51:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 -- start start 2024-04-24T22:51:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 Processor -- start 2024-04-24T22:51:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c28a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a06c28fdc0 con 0x55a06c28d190 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a06c1a1990 con 0x55a06c2828d0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.510+0000 7efdf8e46740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a06c28ff40 con 0x55a06c281fb0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdf8e46740 10 client.0 _ll_register_callbacks cb 0x55a06c1a3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38680/0 (socket says 172.21.15.111:38680) 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdefe00640 1 -- 172.21.15.111:0/730317248 learned_addr learned my addr 172.21.15.111:0/730317248 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c28a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efde7e00640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 -- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 msgr2=0x55a06c28a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 -- 172.21.15.111:0/730317248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 msgr2=0x55a06c28aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdef400640 1 -- 172.21.15.111:0/730317248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06bfc6620 con 0x55a06c281fb0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efdefe00640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.511+0000 7efde7e00640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efdef400640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c28a050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efde000c6a0 tx=0x7efde009b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=587ff20a8dd8b7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00cd200 con 0x55a06c281fb0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7efde009b040 con 0x55a06c281fb0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd8009da0 con 0x55a06c281fb0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efde009bbf0 con 0x55a06c281fb0 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efdf8e46740 10 client.26038 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a06c29bc30 con 0x55a06c281fb0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.512+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde00b3030 con 0x55a06c281fb0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efde0097ad0 con 0x55a06c281fb0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 10 client.26038 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 20 client.26038 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c29d0a0 0x55a06c29f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 4 client.26038 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a06c1f2ff0 con 0x55a06c29d0a0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.513+0000 7efde7e00640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c29d0a0 0x55a06c29f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.514+0000 7efde7e00640 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c29d0a0 0x55a06c29f490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf62e5b40e24e73f server_cookie=7c0d21e994285322 in_seq=0 out_seq=0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.514+0000 7efddcc00640 10 client.26038 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.514+0000 7efddcc00640 1 -- 172.21.15.111:0/730317248 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a06c1f2ff0 con 0x55a06c29d0a0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.514+0000 7efddcc00640 10 client.26038 handle_command_reply: tid=0 2024-04-24T22:51:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 client.26038 shutdown 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 msgr2=0x55a06c28a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c28a050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efde000c6a0 tx=0x7efde009b9c0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 shutdown_connections 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c29d0a0 0x55a06c29f490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c28aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 --2- 172.21.15.111:0/730317248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c28a050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efddcc00640 0 client.26038 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 >> 172.21.15.111:0/730317248 conn(0x55a06c27e4c0 msgr2=0x55a06c284240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 shutdown_connections 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 1 -- 172.21.15.111:0/730317248 wait complete. 2024-04-24T22:51:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.515+0000 7efdf8e46740 20 client.26038 trim_cache size 0 max 16384 2024-04-24T22:51:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.531+0000 7efdf8e46740 1 Processor -- start 2024-04-24T22:51:40.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:40 smithi111 ceph-mon[27746]: pgmap v302: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.531+0000 7efdf8e46740 1 -- start start 2024-04-24T22:51:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c284970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b1e30 con 0x55a06c28d190 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01af700 con 0x55a06c2828d0 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdf8e46740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01ad570 con 0x55a06c281fb0 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38694/0 (socket says 172.21.15.111:38694) 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdefe00640 1 -- 172.21.15.111:0/2556348925 learned_addr learned my addr 172.21.15.111:0/2556348925 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdef400640 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c284970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efde7e00640 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdefe00640 1 -- 172.21.15.111:0/2556348925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 msgr2=0x55a06c284970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.532+0000 7efdefe00640 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c284970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efdefe00640 1 -- 172.21.15.111:0/2556348925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 msgr2=0x55a06c28a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efdefe00640 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efdefe00640 1 -- 172.21.15.111:0/2556348925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06c29faf0 con 0x55a06c28d190 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efdefe00640 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efde802ae00 tx=0x7efde8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6132ae0507ec2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efddf400640 1 -- 172.21.15.111:0/2556348925 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde80305a0 con 0x55a06c28d190 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efddf400640 1 -- 172.21.15.111:0/2556348925 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde8056070 con 0x55a06c28d190 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.533+0000 7efddf400640 1 -- 172.21.15.111:0/2556348925 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8046a50 con 0x55a06c28d190 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- 172.21.15.111:0/2556348925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 msgr2=0x55a06c2a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efde802ae00 tx=0x7efde8045000 comp rx=0 tx=0).stop 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- 172.21.15.111:0/2556348925 shutdown_connections 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c28d190 0x55a06c2a4d60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c2828d0 0x55a06c28a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 --2- 172.21.15.111:0/2556348925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c281fb0 0x55a06c284970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- 172.21.15.111:0/2556348925 >> 172.21.15.111:0/2556348925 conn(0x55a06c27e4c0 msgr2=0x55a06c27d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- 172.21.15.111:0/2556348925 shutdown_connections 2024-04-24T22:51:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- 172.21.15.111:0/2556348925 wait complete. 2024-04-24T22:51:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.534+0000 7efdf8e46740 1 -- start start 2024-04-24T22:51:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 Processor -- start 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c2828d0 0x55a06c287920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c3025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01adbc0 con 0x55a06c281fb0 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01af030 con 0x55a06c28d190 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01ae290 con 0x55a06c2828d0 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdf8e46740 10 client.0 _ll_register_callbacks cb 0x7efde00d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38710/0 (socket says 172.21.15.111:38710) 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.535+0000 7efdef400640 1 -- 172.21.15.111:0/76271871 learned_addr learned my addr 172.21.15.111:0/76271871 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efde7e00640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c2828d0 0x55a06c287920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdefe00640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c3025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 -- 172.21.15.111:0/76271871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c2828d0 msgr2=0x55a06c287920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c2828d0 0x55a06c287920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 -- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 msgr2=0x55a06c3025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c3025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 -- 172.21.15.111:0/76271871 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06c302b10 con 0x55a06c281fb0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdef400640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efde00cf5f0 tx=0x7efde00cf2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5732bd90abad5ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00cd200 con 0x55a06c281fb0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7efde009b250 con 0x55a06c281fb0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.536+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde8056070 con 0x55a06c281fb0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efde009b730 con 0x55a06c281fb0 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 10 client.15990 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a06c303d90 con 0x55a06c281fb0 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde8056070 con 0x55a06c281fb0 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efde009ab40 con 0x55a06c281fb0 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 10 client.15990 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 20 client.15990 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c3044b0 0x55a06c3068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 4 client.15990 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.537+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a06c1ed340 con 0x55a06c3044b0 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.538+0000 7efde7e00640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c3044b0 0x55a06c3068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.538+0000 7efddcc00640 10 client.15990 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.538+0000 7efde7e00640 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c3044b0 0x55a06c3068a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08a8b0fed392b7f server_cookie=3446f27d6a9909b3 in_seq=0 out_seq=0 2024-04-24T22:51:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efddcc00640 1 -- 172.21.15.111:0/76271871 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a06c1ed340 con 0x55a06c3044b0 2024-04-24T22:51:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efddcc00640 10 client.15990 handle_command_reply: tid=0 2024-04-24T22:51:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 client.15990 shutdown 2024-04-24T22:51:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 msgr2=0x55a06c2873e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efde00cf5f0 tx=0x7efde00cf2f0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 shutdown_connections 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55a06c3044b0 0x55a06c3068a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a06c28d190 0x55a06c3025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a06c2828d0 0x55a06c287920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 --2- 172.21.15.111:0/76271871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a06c281fb0 0x55a06c2873e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efddcc00640 0 client.15990 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 >> 172.21.15.111:0/76271871 conn(0x55a06c27e4c0 msgr2=0x55a06c27cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.552+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 shutdown_connections 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf8e46740 1 -- 172.21.15.111:0/76271871 wait complete. 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf8e46740 20 client.15990 trim_cache size 0 max 16384 2024-04-24T22:51:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:51:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:51:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:51:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:51:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 msgr2=0x7efdd40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 0x7efdd40a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efde8005e30 tx=0x7efde803d040 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 msgr2=0x7efdf00fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.553+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efde000e9f0 tx=0x7efde000eeb0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 shutdown_connections 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdf01064d0 0x7efdf00fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efdd40a2d70 0x7efdd40a5230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf0100df0 0x7efdf00fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 --2- 172.21.15.111:0/1494171954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdf0100430 0x7efdf00fc410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 >> 172.21.15.111:0/1494171954 conn(0x7efdf00fbf60 msgr2=0x7efdf0068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 shutdown_connections 2024-04-24T22:51:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:40.554+0000 7efdf4a00640 1 -- 172.21.15.111:0/1494171954 wait complete. 2024-04-24T22:51:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:40 smithi171 ceph-mon[32337]: pgmap v302: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:40 smithi186 ceph-mon[32753]: pgmap v302: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:42.356 INFO:tasks.workunit.client.0.smithi111.stdout:437/600: open fd = 3 2024-04-24T22:51:42.356 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.356 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.356 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.356 INFO:tasks.workunit.client.0.smithi111.stdout:438/600: open fd = 3 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:439/600: open fd = 3 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:440/600: open fd = 3 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:441/600: open fd = 3 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.357 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:442/600: open fd = 3 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:443/600: open fd = 3 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:444/600: open fd = 3 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:445/600: open fd = 3 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.358 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:446/600: open fd = 3 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:447/600: open fd = 3 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:448/600: open fd = 3 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:449/600: open fd = 3 2024-04-24T22:51:42.359 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:450/600: open fd = 3 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:451/600: open fd = 3 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:452/600: open fd = 3 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.360 INFO:tasks.workunit.client.0.smithi111.stdout:453/600: open fd = 3 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:454/600: open fd = 3 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:455/600: open fd = 3 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:456/600: open fd = 3 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.361 INFO:tasks.workunit.client.0.smithi111.stdout:457/600: open fd = 3 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:458/600: open fd = 3 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:459/600: open fd = 3 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:460/600: open fd = 3 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.362 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:461/600: open fd = 3 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:462/600: open fd = 3 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:463/600: open fd = 3 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:464/600: open fd = 3 2024-04-24T22:51:42.363 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:465/600: open fd = 3 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:466/600: open fd = 3 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:467/600: open fd = 3 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.364 INFO:tasks.workunit.client.0.smithi111.stdout:468/600: open fd = 3 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:469/600: open fd = 3 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:470/600: open fd = 3 2024-04-24T22:51:42.365 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:471/600: open fd = 3 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:472/600: open fd = 3 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:473/600: open fd = 3 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.366 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:474/600: open fd = 3 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:475/600: open fd = 3 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:476/600: open fd = 3 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:477/600: open fd = 3 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.367 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:478/600: open fd = 3 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:479/600: open fd = 3 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:480/600: open fd = 3 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:481/600: open fd = 3 2024-04-24T22:51:42.368 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:482/600: open fd = 3 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:483/600: open fd = 3 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:484/600: open fd = 3 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.369 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:485/600: open fd = 3 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:486/600: open fd = 3 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:487/600: open fd = 3 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:488/600: open fd = 3 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.370 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:489/600: open fd = 3 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:490/600: open fd = 3 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.371 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:491/600: open fd = 3 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:492/600: open fd = 3 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.372 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:493/600: open fd = 3 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:494/600: open fd = 3 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:495/600: open fd = 3 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:496/600: open fd = 3 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.373 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:497/600: open fd = 3 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:498/600: open fd = 3 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:51:42.374 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:51:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:42 smithi171 ceph-mon[32337]: pgmap v303: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:51:42.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:42 smithi186 ceph-mon[32753]: pgmap v303: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:51:42.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:42 smithi111 ceph-mon[27746]: pgmap v303: 97 pgs: 97 active+clean; 3.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:51:44.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:44 smithi171 ceph-mon[32337]: pgmap v304: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:44 smithi186 ceph-mon[32753]: pgmap v304: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:44.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:44 smithi111 ceph-mon[27746]: pgmap v304: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:51:46.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:46 smithi171 ceph-mon[32337]: pgmap v305: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:46 smithi186 ceph-mon[32753]: pgmap v305: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:46.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:46 smithi111 ceph-mon[27746]: pgmap v305: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:48.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:48 smithi171 ceph-mon[32337]: pgmap v306: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:48 smithi186 ceph-mon[32753]: pgmap v306: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:48.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:48 smithi111 ceph-mon[27746]: pgmap v306: 97 pgs: 97 active+clean; 3.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:51:50.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:50 smithi171 ceph-mon[32337]: pgmap v307: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:50.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:50 smithi186 ceph-mon[32753]: pgmap v307: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:50 smithi111 ceph-mon[27746]: pgmap v307: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:52.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:52 smithi171 ceph-mon[32337]: pgmap v308: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:52 smithi186 ceph-mon[32753]: pgmap v308: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:52.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:52 smithi111 ceph-mon[27746]: pgmap v308: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:51:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.695+0000 7f586d400640 1 -- 172.21.15.111:0/312630740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 msgr2=0x7f5868108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.695+0000 7f586d400640 1 --2- 172.21.15.111:0/312630740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f5868108380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5858009ca0 tx=0x7f585802a8c0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 -- 172.21.15.111:0/312630740 shutdown_connections 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 --2- 172.21.15.111:0/312630740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f5868114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 --2- 172.21.15.111:0/312630740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5868108950 0x7f586810c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 --2- 172.21.15.111:0/312630740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f5868108380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 -- 172.21.15.111:0/312630740 >> 172.21.15.111:0/312630740 conn(0x7f5868075cd0 msgr2=0x7f58680780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 -- 172.21.15.111:0/312630740 shutdown_connections 2024-04-24T22:51:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.696+0000 7f586d400640 1 -- 172.21.15.111:0/312630740 wait complete. 2024-04-24T22:51:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.697+0000 7f586d400640 1 Processor -- start 2024-04-24T22:51:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.697+0000 7f586d400640 1 -- start start 2024-04-24T22:51:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f586819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5868118150 con 0x7f5868107fa0 2024-04-24T22:51:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5868117fd0 con 0x7f586810d070 2024-04-24T22:51:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f586d400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58681182d0 con 0x7f58681a2e60 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5865e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5867200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5866800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f586819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5865e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:51182/0 (socket says 172.21.15.111:51182) 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5865e00640 1 -- 172.21.15.111:0/3185402709 learned_addr learned my addr 172.21.15.111:0/3185402709 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.698+0000 7f5867200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:54030/0 (socket says 172.21.15.111:54030) 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 -- 172.21.15.111:0/3185402709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 msgr2=0x7f58681a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 -- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 msgr2=0x7f586819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f586819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 -- 172.21.15.111:0/3185402709 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681a3950 con 0x7f586810d070 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5867200640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f5865e00640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5854005fd0 tx=0x7f585400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f585f400640 1 -- 172.21.15.111:0/3185402709 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854017070 con 0x7f586810d070 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681a3c40 con 0x7f586810d070 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f585f400640 1 -- 172.21.15.111:0/3185402709 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854009c30 con 0x7f586810d070 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.699+0000 7f585f400640 1 -- 172.21.15.111:0/3185402709 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854010460 con 0x7f586810d070 2024-04-24T22:51:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.700+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681ab430 con 0x7f586810d070 2024-04-24T22:51:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 Processor -- start 2024-04-24T22:51:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f585f400640 1 -- 172.21.15.111:0/3185402709 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5854003e70 con 0x7f586810d070 2024-04-24T22:51:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 -- start start 2024-04-24T22:51:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561cb4807ab0 con 0x561cb48aefe0 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561cb47b0640 con 0x561cb48a4720 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.703+0000 7f5870213740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561cb4803a70 con 0x561cb48a3e00 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:54032/0 (socket says 172.21.15.111:54032) 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 -- 172.21.15.111:0/1501319318 learned_addr learned my addr 172.21.15.111:0/1501319318 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5865e00640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5866800640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 -- 172.21.15.111:0/1501319318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 msgr2=0x561cb48aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 -- 172.21.15.111:0/1501319318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 msgr2=0x561cb48b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 -- 172.21.15.111:0/1501319318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb451c210 con 0x561cb48a3e00 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5866800640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5865e00640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.704+0000 7f5867200640 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5860011440 tx=0x7f5860011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd5e5236017c1be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5853400640 1 -- 172.21.15.111:0/1501319318 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004020 con 0x561cb48a3e00 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f585f400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 0x7f584c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5853400640 1 -- 172.21.15.111:0/1501319318 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58600041c0 con 0x561cb48a3e00 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5853400640 1 -- 172.21.15.111:0/1501319318 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586000f8e0 con 0x561cb48a3e00 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f585f400640 1 -- 172.21.15.111:0/3185402709 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f58540cf0a0 con 0x7f586810d070 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5866800640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 0x7f584c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5870213740 1 -- 172.21.15.111:0/1501319318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 msgr2=0x561cb48a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.705+0000 7f5870213740 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5860011440 tx=0x7f5860011900 comp rx=0 tx=0).stop 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 -- 172.21.15.111:0/1501319318 shutdown_connections 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 --2- 172.21.15.111:0/1501319318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48a41e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 -- 172.21.15.111:0/1501319318 >> 172.21.15.111:0/1501319318 conn(0x561cb48a0310 msgr2=0x561cb48a06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 -- 172.21.15.111:0/1501319318 shutdown_connections 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 -- 172.21.15.111:0/1501319318 wait complete. 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5866800640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 0x7f584c0a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858009800 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 -- start start 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.706+0000 7f5870213740 1 Processor -- start 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561cb48b1c10 con 0x561cb48aefe0 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561cb47c3990 con 0x561cb48a4720 2024-04-24T22:51:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561cb48b1d90 con 0x561cb48a3e00 2024-04-24T22:51:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5870213740 10 client.0 _ll_register_callbacks cb 0x561cb47c5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5866800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5867200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5866800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:54044/0 (socket says 172.21.15.111:54044) 2024-04-24T22:51:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5866800640 1 -- 172.21.15.111:0/3090152365 learned_addr learned my addr 172.21.15.111:0/3090152365 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.707+0000 7f5865e00640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 -- 172.21.15.111:0/3090152365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 msgr2=0x561cb48abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 -- 172.21.15.111:0/3090152365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 msgr2=0x561cb48aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 -- 172.21.15.111:0/3090152365 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb45e8620 con 0x561cb48a4720 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5865e00640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48ac520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58540cc790 tx=0x7f585409b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81c6b12a3052b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.708+0000 7f5847e00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.709+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58540d1310 con 0x561cb48a4720 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.709+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58600041c0 con 0x561cb48a4720 2024-04-24T22:51:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.709+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f58540cca10 con 0x561cb48a4720 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.709+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f58540ccc10 con 0x561cb48a4720 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 10 client.26012 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561cb48bdb90 con 0x561cb48a4720 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58600041c0 con 0x561cb48a4720 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f58540b4250 con 0x561cb48a4720 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 10 client.26012 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 20 client.26012 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb48bf000 0x561cb48c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 4 client.26012 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.710+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb4814ff0 con 0x561cb48bf000 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.711+0000 7f5866800640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb48bf000 0x561cb48c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.711+0000 7f5866800640 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb48bf000 0x561cb48c13f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d967cff0fb929 server_cookie=eafe45190adf8768 in_seq=0 out_seq=0 2024-04-24T22:51:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.711+0000 7f5847e00640 10 client.26012 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5847e00640 1 -- 172.21.15.111:0/3090152365 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cb4814ff0 con 0x561cb48bf000 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5847e00640 10 client.26012 handle_command_reply: tid=0 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 client.26012 shutdown 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 msgr2=0x561cb48ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48ac520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58540cc790 tx=0x7f585409b330 comp rx=0 tx=0).stop 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 shutdown_connections 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb48bf000 0x561cb48c13f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48ac520 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 --2- 172.21.15.111:0/3090152365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5847e00640 0 client.26012 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 >> 172.21.15.111:0/3090152365 conn(0x561cb48a0310 msgr2=0x561cb48a6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.712+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 shutdown_connections 2024-04-24T22:51:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.713+0000 7f5870213740 1 -- 172.21.15.111:0/3090152365 wait complete. 2024-04-24T22:51:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.713+0000 7f5870213740 20 client.26012 trim_cache size 0 max 16384 2024-04-24T22:51:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 Processor -- start 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 -- start start 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 0x561cb48a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 0x561cb48c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b6390 con 0x561cb48a4720 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b3c60 con 0x561cb48a3e00 2024-04-24T22:51:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5870213740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b1ad0 con 0x561cb48aefe0 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5865e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5865e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48972/0 (socket says 172.21.15.111:48972) 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5865e00640 1 -- 172.21.15.111:0/211165644 learned_addr learned my addr 172.21.15.111:0/211165644 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5866800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 0x561cb48a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.729+0000 7f5867200640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 0x561cb48c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 -- 172.21.15.111:0/211165644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 msgr2=0x561cb48c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 0x561cb48c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 -- 172.21.15.111:0/211165644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 msgr2=0x561cb48a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 0x561cb48a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 -- 172.21.15.111:0/211165644 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb48a6d00 con 0x561cb48a4720 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5866800640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 0x561cb48a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5867200640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 0x561cb48c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5865e00640 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f585409fad0 tx=0x7f58540a41d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec8c03ca06729dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5852a00640 1 -- 172.21.15.111:0/211165644 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585409b0a0 con 0x561cb48a4720 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5852a00640 1 -- 172.21.15.111:0/211165644 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854009de0 con 0x561cb48a4720 2024-04-24T22:51:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.730+0000 7f5852a00640 1 -- 172.21.15.111:0/211165644 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585409a9b0 con 0x561cb48a4720 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- 172.21.15.111:0/211165644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 msgr2=0x561cb48ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f585409fad0 tx=0x7f58540a41d0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- 172.21.15.111:0/211165644 shutdown_connections 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48aefe0 0x561cb48c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48a4720 0x561cb48ac6b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 --2- 172.21.15.111:0/211165644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a3e00 0x561cb48a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- 172.21.15.111:0/211165644 >> 172.21.15.111:0/211165644 conn(0x561cb48a0310 msgr2=0x561cb48abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- 172.21.15.111:0/211165644 shutdown_connections 2024-04-24T22:51:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- 172.21.15.111:0/211165644 wait complete. 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.731+0000 7f5870213740 1 -- start start 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 Processor -- start 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b2120 con 0x561cb48aefe0 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b3590 con 0x561cb48a4720 2024-04-24T22:51:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b27f0 con 0x561cb48a3e00 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5870213740 10 client.0 _ll_register_callbacks cb 0x7f58540d2860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5867200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5866800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5867200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48982/0 (socket says 172.21.15.111:48982) 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5866800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:54068/0 (socket says 172.21.15.111:54068) 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5867200640 1 -- 172.21.15.111:0/2013286875 learned_addr learned my addr 172.21.15.111:0/2013286875 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.732+0000 7f5865e00640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 -- 172.21.15.111:0/2013286875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 msgr2=0x561cb48b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 -- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 msgr2=0x561cb48b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 -- 172.21.15.111:0/2013286875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb48b3da0 con 0x561cb48aefe0 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5866800640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5867200640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5860005f10 tx=0x7f5860005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb264541ae83983d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5847e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:51:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586001d040 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5854009de0 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.733+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f5860003800 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5860031050 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 10 client.16008 fetch_fsmap learned FSMap version 24 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561cb4926100 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854009de0 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5860022430 con 0x561cb48aefe0 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 10 client.16008 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:51:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 20 client.16008 populate_metadata read hostname 'smithi111' 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb49266f0 0x561cb4928b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 4 client.16008 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.734+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cb480f340 con 0x561cb49266f0 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.735+0000 7f5866800640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb49266f0 0x561cb4928b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.735+0000 7f5866800640 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb49266f0 0x561cb4928b10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d58c2a1a742f1a6 server_cookie=92567d890f3fc640 in_seq=0 out_seq=0 2024-04-24T22:51:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.735+0000 7f5847e00640 10 client.16008 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5847e00640 1 -- 172.21.15.111:0/2013286875 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cb480f340 con 0x561cb49266f0 2024-04-24T22:51:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5847e00640 10 client.16008 handle_command_reply: tid=0 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 client.16008 shutdown 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 msgr2=0x561cb48b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5860005f10 tx=0x7f5860005620 comp rx=0 tx=0).stop 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 shutdown_connections 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x561cb49266f0 0x561cb4928b10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cb48aefe0 0x561cb48b35a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561cb48a4720 0x561cb48b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 --2- 172.21.15.111:0/2013286875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561cb48a3e00 0x561cb48b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5847e00640 0 client.16008 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 >> 172.21.15.111:0/2013286875 conn(0x561cb48a0310 msgr2=0x561cb48ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 shutdown_connections 2024-04-24T22:51:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 1 -- 172.21.15.111:0/2013286875 wait complete. 2024-04-24T22:51:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.749+0000 7f5870213740 20 client.16008 trim_cache size 0 max 16384 2024-04-24T22:51:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:51:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:51:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:51:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 msgr2=0x7f584c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 0x7f584c0a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858009800 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 msgr2=0x7f586819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5854005fd0 tx=0x7f585400dde0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 shutdown_connections 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f584c0a2da0 0x7f584c0a5260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58681a2e60 0x7f58681a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f586810d070 0x7f586819eb30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 --2- 172.21.15.111:0/3185402709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5868107fa0 0x7f586819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 >> 172.21.15.111:0/3185402709 conn(0x7f5868075cd0 msgr2=0x7f5868077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 shutdown_connections 2024-04-24T22:51:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:51:53.750+0000 7f586d400640 1 -- 172.21.15.111:0/3185402709 wait complete. 2024-04-24T22:51:54.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:54 smithi171 ceph-mon[32337]: pgmap v309: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:54.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:54 smithi186 ceph-mon[32753]: pgmap v309: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:54.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:54 smithi111 ceph-mon[27746]: pgmap v309: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:51:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:56 smithi171 ceph-mon[32337]: pgmap v310: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:51:56.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:56 smithi186 ceph-mon[32753]: pgmap v310: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:51:56.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:56 smithi111 ceph-mon[27746]: pgmap v310: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:51:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:51:58 smithi171 ceph-mon[32337]: pgmap v311: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:51:58.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:51:58 smithi186 ceph-mon[32753]: pgmap v311: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:51:58.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:51:58 smithi111 ceph-mon[27746]: pgmap v311: 97 pgs: 97 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-04-24T22:52:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:00 smithi171 ceph-mon[32337]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:52:00.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:00 smithi186 ceph-mon[32753]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:52:00.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:00 smithi111 ceph-mon[27746]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-24T22:52:02.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:02 smithi111 ceph-mon[27746]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:02 smithi171 ceph-mon[32337]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:02.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:02 smithi186 ceph-mon[32753]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:04.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:04 smithi171 ceph-mon[32337]: pgmap v314: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 34 op/s 2024-04-24T22:52:04.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:04 smithi186 ceph-mon[32753]: pgmap v314: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 34 op/s 2024-04-24T22:52:04.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:04 smithi111 ceph-mon[27746]: pgmap v314: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 34 op/s 2024-04-24T22:52:06.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:06 smithi171 ceph-mon[32337]: pgmap v315: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 29 op/s 2024-04-24T22:52:06.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:06 smithi186 ceph-mon[32753]: pgmap v315: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 29 op/s 2024-04-24T22:52:06.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:06 smithi111 ceph-mon[27746]: pgmap v315: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 29 op/s 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 -- 172.21.15.111:0/639263687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42741125d0 msgr2=0x7f42741149c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 --2- 172.21.15.111:0/639263687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42741125d0 0x7f42741149c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f426c009890 tx=0x7f426c02a370 comp rx=0 tx=0).stop 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 -- 172.21.15.111:0/639263687 shutdown_connections 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 --2- 172.21.15.111:0/639263687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42741125d0 0x7f42741149c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 --2- 172.21.15.111:0/639263687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4274076590 0x7f4274112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 --2- 172.21.15.111:0/639263687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 0x7f4274076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 -- 172.21.15.111:0/639263687 >> 172.21.15.111:0/639263687 conn(0x7f42740ff440 msgr2=0x7f4274101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 -- 172.21.15.111:0/639263687 shutdown_connections 2024-04-24T22:52:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.889+0000 7f4279a00640 1 -- 172.21.15.111:0/639263687 wait complete. 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.890+0000 7f4279a00640 1 Processor -- start 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.890+0000 7f4279a00640 1 -- start start 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 0x7f427419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 0x7f427419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 0x7f42741a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42741180f0 con 0x7f4274075c70 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4274117f70 con 0x7f4274076590 2024-04-24T22:52:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4279a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4274118270 con 0x7f42741125d0 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4272a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 0x7f427419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4272000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 0x7f427419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4273400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 0x7f42741a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4272000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 0x7f427419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:37132/0 (socket says 172.21.15.111:37132) 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.891+0000 7f4272000640 1 -- 172.21.15.111:0/4019147365 learned_addr learned my addr 172.21.15.111:0/4019147365 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 -- 172.21.15.111:0/4019147365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 msgr2=0x7f427419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 0x7f427419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 -- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 msgr2=0x7f427419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 0x7f427419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 -- 172.21.15.111:0/4019147365 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42741a3520 con 0x7f42741125d0 2024-04-24T22:52:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4273400640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 0x7f42741a2df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f426c02a890 tx=0x7f426c033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f426b400640 1 -- 172.21.15.111:0/4019147365 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c037040 con 0x7f42741125d0 2024-04-24T22:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42741a37b0 con 0x7f42741125d0 2024-04-24T22:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741ab0a0 con 0x7f42741125d0 2024-04-24T22:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f426b400640 1 -- 172.21.15.111:0/4019147365 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c033cd0 con 0x7f42741125d0 2024-04-24T22:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.892+0000 7f426b400640 1 -- 172.21.15.111:0/4019147365 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c003920 con 0x7f42741125d0 2024-04-24T22:52:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.896+0000 7f427c853740 1 Processor -- start 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f426b400640 1 -- 172.21.15.111:0/4019147365 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f426c042020 con 0x7f42741125d0 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 -- start start 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da1774c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da17756fe0 0x55da177593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55da176afab0 con 0x55da17756fe0 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55da17658640 con 0x55da1774c720 2024-04-24T22:52:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.897+0000 7f427c853740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55da176aba70 con 0x55da1774be00 2024-04-24T22:52:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da1774c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:37140/0 (socket says 172.21.15.111:37140) 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 -- 172.21.15.111:0/738248835 learned_addr learned my addr 172.21.15.111:0/738248835 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272000640 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da17756fe0 0x55da177593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 -- 172.21.15.111:0/738248835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 msgr2=0x55da1774c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da1774c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 -- 172.21.15.111:0/738248835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da17756fe0 msgr2=0x55da177593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da17756fe0 0x55da177593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 -- 172.21.15.111:0/738248835 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da173c4210 con 0x55da1774c720 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272000640 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da1774c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f426b400640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 0x7f42580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.898+0000 7f4272a00640 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f425c009cf0 tx=0x7f425c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df3dd84f2d977d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f426b400640 1 -- 172.21.15.111:0/4019147365 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f426c0eeaa0 con 0x7f42741125d0 2024-04-24T22:52:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f4263e00640 1 -- 172.21.15.111:0/738248835 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c004230 con 0x55da1774c720 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f4272000640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 0x7f42580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f4263e00640 1 -- 172.21.15.111:0/738248835 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425c0043d0 con 0x55da1774c720 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f4263e00640 1 -- 172.21.15.111:0/738248835 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0198c0 con 0x55da1774c720 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f427c853740 1 -- 172.21.15.111:0/738248835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 msgr2=0x55da17756aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f425c009cf0 tx=0x7f425c011810 comp rx=0 tx=0).stop 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.899+0000 7f4272000640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 0x7f42580a52e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55da17749730 tx=0x7f42640095c0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 -- 172.21.15.111:0/738248835 shutdown_connections 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da17756fe0 0x55da177593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774c720 0x55da17756aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 --2- 172.21.15.111:0/738248835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da1774c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 -- 172.21.15.111:0/738248835 >> 172.21.15.111:0/738248835 conn(0x55da17748310 msgr2=0x55da177486f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 -- 172.21.15.111:0/738248835 shutdown_connections 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 -- 172.21.15.111:0/738248835 wait complete. 2024-04-24T22:52:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 -- start start 2024-04-24T22:52:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.900+0000 7f427c853740 1 Processor -- start 2024-04-24T22:52:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da177544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da17754a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55da17759c10 con 0x55da1774c720 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55da1766b990 con 0x55da17756fe0 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55da17759d90 con 0x55da1774be00 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f427c853740 10 client.0 _ll_register_callbacks cb 0x55da1766d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f4272a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f4272000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da177544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f4272a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:41854/0 (socket says 172.21.15.111:41854) 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f4272a00640 1 -- 172.21.15.111:0/15920887 learned_addr learned my addr 172.21.15.111:0/15920887 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.901+0000 7f4273400640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da17754a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 -- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 msgr2=0x55da17754a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da17754a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 -- 172.21.15.111:0/15920887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 msgr2=0x55da177544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da177544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 -- 172.21.15.111:0/15920887 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da17490620 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4272a00640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f425c005e00 tx=0x7f425c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20806d6a9d1dc4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4260c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c027070 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f425c005240 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.902+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425c0043d0 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f425c033050 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 10 client.26074 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da17765b00 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425c0043d0 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f425c0195c0 con 0x55da1774be00 2024-04-24T22:52:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 10 client.26074 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 20 client.26074 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da17766f70 0x55da17769360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 4 client.26074 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.903+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da176bcff0 con 0x55da17766f70 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.904+0000 7f4272000640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da17766f70 0x55da17769360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.904+0000 7f4272000640 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da17766f70 0x55da17769360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676f2d614a5d7361 server_cookie=c5ad2d80425b0241 in_seq=0 out_seq=0 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.904+0000 7f4260c00640 10 client.26074 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f4260c00640 1 -- 172.21.15.111:0/15920887 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da176bcff0 con 0x55da17766f70 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f4260c00640 10 client.26074 handle_command_reply: tid=0 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 client.26074 shutdown 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 msgr2=0x55da17753f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f425c005e00 tx=0x7f425c003820 comp rx=0 tx=0).stop 2024-04-24T22:52:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 shutdown_connections 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da17766f70 0x55da17769360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da17754a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da177544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 --2- 172.21.15.111:0/15920887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17753f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 >> 172.21.15.111:0/15920887 conn(0x55da17748310 msgr2=0x55da1774e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.905+0000 7f4260c00640 0 client.26074 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.906+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 shutdown_connections 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.906+0000 7f427c853740 1 -- 172.21.15.111:0/15920887 wait complete. 2024-04-24T22:52:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.906+0000 7f427c853740 20 client.26074 trim_cache size 0 max 16384 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 Processor -- start 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 -- start start 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774be00 0x55da17747300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da1776ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b6250 con 0x55da1774c720 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b3b20 con 0x55da1774be00 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.922+0000 7f427c853740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b1990 con 0x55da17756fe0 2024-04-24T22:52:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:41864/0 (socket says 172.21.15.111:41864) 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 -- 172.21.15.111:0/1128287899 learned_addr learned my addr 172.21.15.111:0/1128287899 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4272000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da1776ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4272a00640 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774be00 0x55da17747300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 -- 172.21.15.111:0/1128287899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774be00 msgr2=0x55da17747300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774be00 0x55da17747300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 -- 172.21.15.111:0/1128287899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 msgr2=0x55da1776ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da1776ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 -- 172.21.15.111:0/1128287899 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1774c550 con 0x55da17756fe0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4273400640 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f426c009250 tx=0x7f426c033f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b7904b7be679e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.923+0000 7f4263400640 1 -- 172.21.15.111:0/1128287899 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c0bb050 con 0x55da17756fe0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f4263400640 1 -- 172.21.15.111:0/1128287899 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c0bf400 con 0x55da17756fe0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f4263400640 1 -- 172.21.15.111:0/1128287899 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c0b76c0 con 0x55da17756fe0 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 -- 172.21.15.111:0/1128287899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 msgr2=0x55da1776f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f426c009250 tx=0x7f426c033f00 comp rx=0 tx=0).stop 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 -- 172.21.15.111:0/1128287899 shutdown_connections 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da17756fe0 0x55da1776f010 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da1776ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 --2- 172.21.15.111:0/1128287899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da1774be00 0x55da17747300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 -- 172.21.15.111:0/1128287899 >> 172.21.15.111:0/1128287899 conn(0x55da17748310 msgr2=0x55da17746bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 -- 172.21.15.111:0/1128287899 shutdown_connections 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.924+0000 7f427c853740 1 -- 172.21.15.111:0/1128287899 wait complete. 2024-04-24T22:52:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 -- start start 2024-04-24T22:52:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 Processor -- start 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da17751280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da177517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b1fe0 con 0x55da1774c720 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b3450 con 0x55da17756fe0 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b26b0 con 0x55da1774be00 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.925+0000 7f427c853740 10 client.0 _ll_register_callbacks cb 0x7f425c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:41880/0 (socket says 172.21.15.111:41880) 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 -- 172.21.15.111:0/2021730158 learned_addr learned my addr 172.21.15.111:0/2021730158 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272000640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da17751280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4273400640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da177517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 -- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 msgr2=0x55da177517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da177517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 -- 172.21.15.111:0/2021730158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 msgr2=0x55da17751280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da17751280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 -- 172.21.15.111:0/2021730158 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da177cc410 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4272a00640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f425c005e00 tx=0x7f425c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d1e8f3d8e3a192 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4260c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c03c060 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c0bf400 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.926+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f425c019750 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f425c027040 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 10 client.26080 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da177cd720 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c0bf400 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f425c004660 con 0x55da1774be00 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 10 client.26080 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 20 client.26080 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da177cddb0 0x55da177d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 4 client.26080 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.927+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da176b7340 con 0x55da177cddb0 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.928+0000 7f4272000640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da177cddb0 0x55da177d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.928+0000 7f4272000640 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da177cddb0 0x55da177d01d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4423db488b94c8f3 server_cookie=29d17c67c9379a9c in_seq=0 out_seq=0 2024-04-24T22:52:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.928+0000 7f4260c00640 10 client.26080 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.942+0000 7f4260c00640 1 -- 172.21.15.111:0/2021730158 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da176b7340 con 0x55da177cddb0 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.942+0000 7f4260c00640 10 client.26080 handle_command_reply: tid=0 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.942+0000 7f427c853740 1 client.26080 shutdown 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 msgr2=0x55da17750d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f425c005e00 tx=0x7f425c005d30 comp rx=0 tx=0).stop 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 shutdown_connections 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55da177cddb0 0x55da177d01d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da17756fe0 0x55da177517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da1774c720 0x55da17751280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f4260c00640 0 client.26080 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 --2- 172.21.15.111:0/2021730158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da1774be00 0x55da17750d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 >> 172.21.15.111:0/2021730158 conn(0x55da17748310 msgr2=0x55da17746bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 shutdown_connections 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 1 -- 172.21.15.111:0/2021730158 wait complete. 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.943+0000 7f427c853740 20 client.26080 trim_cache size 0 max 16384 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:52:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 msgr2=0x7f42580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 0x7f42580a52e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55da17749730 tx=0x7f42640095c0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 msgr2=0x7f42741a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 0x7f42741a2df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f426c02a890 tx=0x7f426c033ae0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 shutdown_connections 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f42580a2e20 0x7f42580a52e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42741125d0 0x7f42741a2df0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4274076590 0x7f427419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 --2- 172.21.15.111:0/4019147365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274075c70 0x7f427419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 >> 172.21.15.111:0/4019147365 conn(0x7f42740ff440 msgr2=0x7f4274101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 shutdown_connections 2024-04-24T22:52:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:06.944+0000 7f4279a00640 1 -- 172.21.15.111:0/4019147365 wait complete. 2024-04-24T22:52:07.658 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:07 smithi171 ceph-mon[32337]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T22:52:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:07 smithi186 ceph-mon[32753]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T22:52:07.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:07 smithi111 ceph-mon[27746]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T22:52:08.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:08 smithi171 ceph-mon[32337]: pgmap v317: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:52:08.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:08 smithi186 ceph-mon[32753]: pgmap v317: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:52:08.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:08 smithi111 ceph-mon[27746]: pgmap v317: 97 pgs: 97 active+clean; 4.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-04-24T22:52:10.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:10 smithi171 ceph-mon[32337]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:10.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:10 smithi186 ceph-mon[32753]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:10.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:10 smithi111 ceph-mon[27746]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:12.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:12 smithi171 ceph-mon[32337]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:12.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:12 smithi171 ceph-mon[32337]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T22:52:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:12 smithi186 ceph-mon[32753]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:12 smithi186 ceph-mon[32753]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T22:52:12.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:12 smithi111 ceph-mon[27746]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-04-24T22:52:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:12 smithi111 ceph-mon[27746]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T22:52:14.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:14 smithi171 ceph-mon[32337]: pgmap v321: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:52:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:14 smithi186 ceph-mon[32753]: pgmap v321: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:52:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:14 smithi111 ceph-mon[27746]: pgmap v321: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-04-24T22:52:16.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:16 smithi171 ceph-mon[32337]: pgmap v322: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-24T22:52:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:16 smithi186 ceph-mon[32753]: pgmap v322: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-24T22:52:16.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:16 smithi111 ceph-mon[27746]: pgmap v322: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-24T22:52:18.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:18 smithi171 ceph-mon[32337]: pgmap v323: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:52:18.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:18 smithi186 ceph-mon[32753]: pgmap v323: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:52:18.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:18 smithi111 ceph-mon[27746]: pgmap v323: 97 pgs: 97 active+clean; 4.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 -- 172.21.15.111:0/2435531889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940103920 msgr2=0x7fb940103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 --2- 172.21.15.111:0/2435531889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940103920 0x7fb940103da0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb928009930 tx=0x7fb92802a440 comp rx=0 tx=0).stop 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 -- 172.21.15.111:0/2435531889 shutdown_connections 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 --2- 172.21.15.111:0/2435531889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb9401107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 --2- 172.21.15.111:0/2435531889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940103920 0x7fb940103da0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 --2- 172.21.15.111:0/2435531889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940109930 0x7fb940109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.087+0000 7fb947200640 1 -- 172.21.15.111:0/2435531889 >> 172.21.15.111:0/2435531889 conn(0x7fb9400ff430 msgr2=0x7fb940101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.088+0000 7fb947200640 1 -- 172.21.15.111:0/2435531889 shutdown_connections 2024-04-24T22:52:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.088+0000 7fb947200640 1 -- 172.21.15.111:0/2435531889 wait complete. 2024-04-24T22:52:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 Processor -- start 2024-04-24T22:52:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 -- start start 2024-04-24T22:52:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 0x7fb94019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940109930 0x7fb94019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb940113d90 con 0x7fb940109930 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb940113c10 con 0x7fb940103920 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.089+0000 7fb947200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb940113f10 con 0x7fb9401042e0 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 0x7fb94019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52458/0 (socket says 172.21.15.111:52458) 2024-04-24T22:52:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 -- 172.21.15.111:0/1367644376 learned_addr learned my addr 172.21.15.111:0/1367644376 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb945200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940109930 0x7fb94019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 -- 172.21.15.111:0/1367644376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 msgr2=0x7fb94019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 0x7fb94019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 -- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940109930 msgr2=0x7fb94019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940109930 0x7fb94019ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb937e00640 1 -- 172.21.15.111:0/1367644376 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94019f150 con 0x7fb9401042e0 2024-04-24T22:52:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.090+0000 7fb93fe00640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 0x7fb94019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb937e00640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb92802a960 tx=0x7fb928003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb93d600640 1 -- 172.21.15.111:0/1367644376 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92803f070 con 0x7fb9401042e0 2024-04-24T22:52:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94019f3e0 con 0x7fb9401042e0 2024-04-24T22:52:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb93d600640 1 -- 172.21.15.111:0/1367644376 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb928031c90 con 0x7fb9401042e0 2024-04-24T22:52:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb93d600640 1 -- 172.21.15.111:0/1367644376 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928035030 con 0x7fb9401042e0 2024-04-24T22:52:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.091+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9401a6cd0 con 0x7fb9401042e0 2024-04-24T22:52:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.094+0000 7fb94a1d2740 1 Processor -- start 2024-04-24T22:52:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.094+0000 7fb94a1d2740 1 -- start start 2024-04-24T22:52:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.094+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe21bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.094+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562fe211fab0 con 0x562fe21bbe00 2024-04-24T22:52:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562fe20c8640 con 0x562fe21bc720 2024-04-24T22:52:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562fe211ba70 con 0x562fe21c6fe0 2024-04-24T22:52:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93d600640 1 -- 172.21.15.111:0/1367644376 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb928042070 con 0x7fb9401042e0 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb945200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe21bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57402/0 (socket says 172.21.15.111:57402) 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 -- 172.21.15.111:0/1042248745 learned_addr learned my addr 172.21.15.111:0/1042248745 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb937e00640 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 -- 172.21.15.111:0/1042248745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 msgr2=0x562fe21c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 -- 172.21.15.111:0/1042248745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 msgr2=0x562fe21bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe21bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb93fe00640 1 -- 172.21.15.111:0/1042248745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe1e34210 con 0x562fe21bc720 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.095+0000 7fb945200640 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe21bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.096+0000 7fb93fe00640 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9300114c0 tx=0x7fb930011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd28ebddde8a3b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.096+0000 7fb935600640 1 -- 172.21.15.111:0/1042248745 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93000f5c0 con 0x562fe21bc720 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.096+0000 7fb935600640 1 -- 172.21.15.111:0/1042248745 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93000fc40 con 0x562fe21bc720 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.096+0000 7fb935600640 1 -- 172.21.15.111:0/1042248745 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930019cc0 con 0x562fe21bc720 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb93d600640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 0x7fb9240a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 -- 172.21.15.111:0/1042248745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 msgr2=0x562fe21c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9300114c0 tx=0x7fb930011980 comp rx=0 tx=0).stop 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 -- 172.21.15.111:0/1042248745 shutdown_connections 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb93d600640 1 -- 172.21.15.111:0/1367644376 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fb9280ef9d0 con 0x7fb9401042e0 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bc720 0x562fe21c6aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/1042248745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe21bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 -- 172.21.15.111:0/1042248745 >> 172.21.15.111:0/1042248745 conn(0x562fe21b8310 msgr2=0x562fe21b86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb945200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 0x7fb9240a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 -- 172.21.15.111:0/1042248745 shutdown_connections 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.097+0000 7fb94a1d2740 1 -- 172.21.15.111:0/1042248745 wait complete. 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.098+0000 7fb94a1d2740 1 -- start start 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.098+0000 7fb945200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 0x7fb9240a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb938009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.098+0000 7fb94a1d2740 1 Processor -- start 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562fe21c9c10 con 0x562fe21bc720 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562fe20db990 con 0x562fe21bbe00 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562fe21c9d90 con 0x562fe21c6fe0 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb94a1d2740 10 client.0 _ll_register_callbacks cb 0x562fe20dd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57404/0 (socket says 172.21.15.111:57404) 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 -- 172.21.15.111:0/2185040368 learned_addr learned my addr 172.21.15.111:0/2185040368 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb937e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb945200640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 -- 172.21.15.111:0/2185040368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 msgr2=0x562fe21c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 -- 172.21.15.111:0/2185040368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 msgr2=0x562fe21c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.099+0000 7fb93fe00640 1 -- 172.21.15.111:0/2185040368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe1f00620 con 0x562fe21bbe00 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb930019800 tx=0x7fb930009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3917412fe71525bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb92e000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930026020 con 0x562fe21bbe00 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fb93000f5c0 con 0x562fe21bbe00 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93000fc40 con 0x562fe21bbe00 2024-04-24T22:52:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb930031050 con 0x562fe21bbe00 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb94a1d2740 10 client.26048 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562fe21d5b00 con 0x562fe21bbe00 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.100+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93000fc40 con 0x562fe21bbe00 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb9300047e0 con 0x562fe21bbe00 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 10 client.26048 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 20 client.26048 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe21d6f60 0x562fe21d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 4 client.26048 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe212cff0 con 0x562fe21d6f60 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.101+0000 7fb937e00640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe21d6f60 0x562fe21d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.102+0000 7fb937e00640 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe21d6f60 0x562fe21d9350 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1885c0a5083e7980 server_cookie=b07549d606b29c0d in_seq=0 out_seq=0 2024-04-24T22:52:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.102+0000 7fb92e000640 10 client.26048 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb92e000640 1 -- 172.21.15.111:0/2185040368 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562fe212cff0 con 0x562fe21d6f60 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb92e000640 10 client.26048 handle_command_reply: tid=0 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 client.26048 shutdown 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 msgr2=0x562fe21c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb930019800 tx=0x7fb930009db0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 shutdown_connections 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe21d6f60 0x562fe21d9350 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb92e000640 0 client.26048 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2185040368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21c3f90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 >> 172.21.15.111:0/2185040368 conn(0x562fe21b8310 msgr2=0x562fe21be090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 shutdown_connections 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2185040368 wait complete. 2024-04-24T22:52:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.103+0000 7fb94a1d2740 20 client.26048 trim_cache size 0 max 16384 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.119+0000 7fb94a1d2740 1 Processor -- start 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.119+0000 7fb94a1d2740 1 -- start start 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401b1e90 con 0x562fe21bc720 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401af760 con 0x562fe21bbe00 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401ad5d0 con 0x562fe21c6fe0 2024-04-24T22:52:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:57416/0 (socket says 172.21.15.111:57416) 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 -- 172.21.15.111:0/3225350884 learned_addr learned my addr 172.21.15.111:0/3225350884 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb937e00640 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb945200640 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 -- 172.21.15.111:0/3225350884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 msgr2=0x562fe21deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21deb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 -- 172.21.15.111:0/3225350884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 msgr2=0x562fe21b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.120+0000 7fb93fe00640 1 -- 172.21.15.111:0/3225350884 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe21d96c0 con 0x562fe21bbe00 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.121+0000 7fb93fe00640 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9300224f0 tx=0x7fb930009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f42f79d8e18dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.121+0000 7fb934c00640 1 -- 172.21.15.111:0/3225350884 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930026040 con 0x562fe21bbe00 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.121+0000 7fb934c00640 1 -- 172.21.15.111:0/3225350884 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb930005520 con 0x562fe21bbe00 2024-04-24T22:52:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.121+0000 7fb934c00640 1 -- 172.21.15.111:0/3225350884 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300035f0 con 0x562fe21bbe00 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- 172.21.15.111:0/3225350884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 msgr2=0x562fe21d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9300224f0 tx=0x7fb930009d40 comp rx=0 tx=0).stop 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- 172.21.15.111:0/3225350884 shutdown_connections 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21c6fe0 0x562fe21deb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bc720 0x562fe21b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/3225350884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21bbe00 0x562fe21d2ad0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- 172.21.15.111:0/3225350884 >> 172.21.15.111:0/3225350884 conn(0x562fe21b8310 msgr2=0x562fe21b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- 172.21.15.111:0/3225350884 shutdown_connections 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- 172.21.15.111:0/3225350884 wait complete. 2024-04-24T22:52:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 -- start start 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.122+0000 7fb94a1d2740 1 Processor -- start 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe223e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21c6fe0 0x562fe21ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401adc20 con 0x562fe21bbe00 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401af090 con 0x562fe21c6fe0 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401ae2f0 con 0x562fe21bc720 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb94a1d2740 10 client.0 _ll_register_callbacks cb 0x7fb938009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe223e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb937e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb937e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:52474/0 (socket says 172.21.15.111:52474) 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb937e00640 1 -- 172.21.15.111:0/2098827655 learned_addr learned my addr 172.21.15.111:0/2098827655 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.123+0000 7fb945200640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21c6fe0 0x562fe21ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 -- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21c6fe0 msgr2=0x562fe21ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21c6fe0 0x562fe21ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 -- 172.21.15.111:0/2098827655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 msgr2=0x562fe223e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe223e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 -- 172.21.15.111:0/2098827655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe21cab90 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe223e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb937e00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb928005e90 tx=0x7fb92802a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feed2e348f3851f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb92e000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928049040 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fb9280ed3a0 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb930005520 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.124+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb9280c02a0 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 10 client.26098 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562fe2245890 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb930005520 con 0x562fe21bc720 2024-04-24T22:52:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb9280c4760 con 0x562fe21bc720 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 10 client.26098 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 20 client.26098 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe2245fc0 0x562fe22483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 4 client.26098 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fe2127340 con 0x562fe2245fc0 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.125+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe2245fc0 0x562fe22483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.126+0000 7fb93fe00640 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe2245fc0 0x562fe22483e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f1bffc6520088c server_cookie=b1353f2178b52725 in_seq=0 out_seq=0 2024-04-24T22:52:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.126+0000 7fb92e000640 10 client.26098 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.140+0000 7fb92e000640 1 -- 172.21.15.111:0/2098827655 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fe2127340 con 0x562fe2245fc0 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.140+0000 7fb92e000640 10 client.26098 handle_command_reply: tid=0 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.140+0000 7fb94a1d2740 1 client.26098 shutdown 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 msgr2=0x562fe223e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb928005e90 tx=0x7fb92802a440 comp rx=0 tx=0).stop 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 shutdown_connections 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x562fe2245fc0 0x562fe22483e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562fe21c6fe0 0x562fe21ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fe21bc720 0x562fe223e5e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 --2- 172.21.15.111:0/2098827655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fe21bbe00 0x562fe223e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb92e000640 0 client.26098 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 >> 172.21.15.111:0/2098827655 conn(0x562fe21b8310 msgr2=0x562fe21b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 shutdown_connections 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 1 -- 172.21.15.111:0/2098827655 wait complete. 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.141+0000 7fb94a1d2740 20 client.26098 trim_cache size 0 max 16384 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:52:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 msgr2=0x7fb9240a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 0x7fb9240a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb938009500 comp rx=0 tx=0).stop 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 msgr2=0x7fb94019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb92802a960 tx=0x7fb928003b20 comp rx=0 tx=0).stop 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 shutdown_connections 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb940109930 0x7fb94019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb9240a2e20 0x7fb9240a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9401042e0 0x7fb94019a6f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 --2- 172.21.15.111:0/1367644376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb940103920 0x7fb94019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 >> 172.21.15.111:0/1367644376 conn(0x7fb9400ff430 msgr2=0x7fb9400ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 shutdown_connections 2024-04-24T22:52:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:20.142+0000 7fb947200640 1 -- 172.21.15.111:0/1367644376 wait complete. 2024-04-24T22:52:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:20 smithi171 ceph-mon[32337]: pgmap v324: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:20 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:20.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:20 smithi186 ceph-mon[32753]: pgmap v324: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:20.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:20 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:20.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:20 smithi111 ceph-mon[27746]: pgmap v324: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:20.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:20 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:21.937 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:21 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.937 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:21 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.937 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:21 smithi171 ceph-mon[32337]: pgmap v325: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:21.937 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:21 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.937 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:21 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:21 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:21 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:21 smithi186 ceph-mon[32753]: pgmap v325: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:21 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:21.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:21 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:22.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:21 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:22.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:21 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:22.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:21 smithi111 ceph-mon[27746]: pgmap v325: 97 pgs: 97 active+clean; 4.9 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-24T22:52:22.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:21 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:22.033 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:21 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.156 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:22 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:22 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:23.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:22 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: pgmap v326: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.157 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:52:24.158 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:23 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:52:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:52:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:52:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: pgmap v326: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-04-24T22:52:24.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:52:24.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:23 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:52:24.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: pgmap v326: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-24T22:52:24.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:23 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-24T22:52:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:24 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:25.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:24 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:25.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:24 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:25.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:24 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:25.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:24 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:25.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:24 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: pgmap v327: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:26.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:25 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: pgmap v327: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:52:26.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:52:26.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:26.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:26.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:25 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:26.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: pgmap v327: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:26.407 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:25 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:52:28.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:27 smithi186 ceph-mon[32753]: pgmap v328: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:28.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:27 smithi111 ceph-mon[27746]: pgmap v328: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:28.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:27 smithi171 ceph-mon[32337]: pgmap v328: 97 pgs: 97 active+clean; 5.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:30.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:29 smithi186 ceph-mon[32753]: pgmap v329: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:30.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:29 smithi111 ceph-mon[27746]: pgmap v329: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:30.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:29 smithi171 ceph-mon[32337]: pgmap v329: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:32.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:31 smithi186 ceph-mon[32753]: pgmap v330: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:32.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:31 smithi111 ceph-mon[27746]: pgmap v330: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:32.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:31 smithi171 ceph-mon[32337]: pgmap v330: 97 pgs: 97 active+clean; 5.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 -- 172.21.15.111:0/3977880549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8109930 msgr2=0x7f63c8109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 --2- 172.21.15.111:0/3977880549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8109930 0x7f63c8109d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009860 tx=0x7f63b802a340 comp rx=0 tx=0).stop 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 -- 172.21.15.111:0/3977880549 shutdown_connections 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 --2- 172.21.15.111:0/3977880549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 0x7f63c81107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 --2- 172.21.15.111:0/3977880549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8103920 0x7f63c8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 --2- 172.21.15.111:0/3977880549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8109930 0x7f63c8109d10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.286+0000 7f63cd600640 1 -- 172.21.15.111:0/3977880549 >> 172.21.15.111:0/3977880549 conn(0x7f63c80ff430 msgr2=0x7f63c8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.287+0000 7f63cd600640 1 -- 172.21.15.111:0/3977880549 shutdown_connections 2024-04-24T22:52:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.287+0000 7f63cd600640 1 -- 172.21.15.111:0/3977880549 wait complete. 2024-04-24T22:52:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.287+0000 7f63cd600640 1 Processor -- start 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 -- start start 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 0x7f63c819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 0x7f63c819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113d40 con 0x7f63c8109930 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113bc0 con 0x7f63c8103920 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63cd600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113ec0 con 0x7f63c81042e0 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47172/0 (socket says 172.21.15.111:47172) 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63c7400640 1 -- 172.21.15.111:0/1746412718 learned_addr learned my addr 172.21.15.111:0/1746412718 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.288+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 0x7f63c819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c6000640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 0x7f63c819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 -- 172.21.15.111:0/1746412718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 msgr2=0x7f63c819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 0x7f63c819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 -- 172.21.15.111:0/1746412718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 msgr2=0x7f63c819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 0x7f63c819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 -- 172.21.15.111:0/1746412718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819f110 con 0x7f63c8109930 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 0x7f63c819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.289+0000 7f63c7400640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009fd0 tx=0x7f63c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.290+0000 7f63bf400640 1 -- 172.21.15.111:0/1746412718 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004b20 con 0x7f63c8109930 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.290+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819f400 con 0x7f63c8109930 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.290+0000 7f63bf400640 1 -- 172.21.15.111:0/1746412718 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c0003e50 con 0x7f63c8109930 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.290+0000 7f63bf400640 1 -- 172.21.15.111:0/1746412718 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c000b630 con 0x7f63c8109930 2024-04-24T22:52:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.290+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a6c90 con 0x7f63c8109930 2024-04-24T22:52:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.293+0000 7f63d0588740 1 Processor -- start 2024-04-24T22:52:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.293+0000 7f63bf400640 1 -- 172.21.15.111:0/1746412718 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f63c001d020 con 0x7f63c8109930 2024-04-24T22:52:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.293+0000 7f63d0588740 1 -- start start 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d050aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d0533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56179cfa9ab0 con 0x56179d046720 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56179cf52640 con 0x56179d045e00 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63d0588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56179cfa5a70 con 0x56179d050fe0 2024-04-24T22:52:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:40066/0 (socket says 172.21.15.111:40066) 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 -- 172.21.15.111:0/696351703 learned_addr learned my addr 172.21.15.111:0/696351703 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6a00640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d050aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d0533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 -- 172.21.15.111:0/696351703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 msgr2=0x56179d0533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d0533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 -- 172.21.15.111:0/696351703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 msgr2=0x56179d050aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d050aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6000640 1 -- 172.21.15.111:0/696351703 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179ccbe210 con 0x56179d045e00 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.294+0000 7f63c6a00640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d050aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63c6000640 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63b40058c0 tx=0x7f63b4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97da7fc6b48e2bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63bf400640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 0x7f63ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63b3400640 1 -- 172.21.15.111:0/696351703 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4004050 con 0x56179d045e00 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63b3400640 1 -- 172.21.15.111:0/696351703 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b4005500 con 0x56179d045e00 2024-04-24T22:52:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63b3400640 1 -- 172.21.15.111:0/696351703 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4004d00 con 0x56179d045e00 2024-04-24T22:52:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63bf400640 1 -- 172.21.15.111:0/1746412718 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f63c00cf3a0 con 0x7f63c8109930 2024-04-24T22:52:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.295+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 0x7f63ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 0x7f63ac0a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63b802a930 tx=0x7f63b80056d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 -- 172.21.15.111:0/696351703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 msgr2=0x56179d0461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63b40058c0 tx=0x7f63b4005850 comp rx=0 tx=0).stop 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 -- 172.21.15.111:0/696351703 shutdown_connections 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d0533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d050aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 --2- 172.21.15.111:0/696351703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d0461e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 -- 172.21.15.111:0/696351703 >> 172.21.15.111:0/696351703 conn(0x56179d042310 msgr2=0x56179d0426f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 -- 172.21.15.111:0/696351703 shutdown_connections 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.296+0000 7f63d0588740 1 -- 172.21.15.111:0/696351703 wait complete. 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.297+0000 7f63d0588740 1 -- start start 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.297+0000 7f63d0588740 1 Processor -- start 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d04e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d04e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56179d053c10 con 0x56179d045e00 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56179cf65990 con 0x56179d050fe0 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56179d053d90 con 0x56179d046720 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63d0588740 10 client.0 _ll_register_callbacks cb 0x56179cf67950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47192/0 (socket says 172.21.15.111:47192) 2024-04-24T22:52:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63c6a00640 1 -- 172.21.15.111:0/848609190 learned_addr learned my addr 172.21.15.111:0/848609190 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63c6000640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d04e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.298+0000 7f63c7400640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d04e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 -- 172.21.15.111:0/848609190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 msgr2=0x56179d04e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d04e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 -- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 msgr2=0x56179d04e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d04e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 -- 172.21.15.111:0/848609190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179cd8a620 con 0x56179d045e00 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63c6a00640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f63b80057c0 tx=0x7f63b802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f170dd628c0e02d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b804a020 con 0x56179d045e00 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f63b8003970 con 0x56179d045e00 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b4005500 con 0x56179d045e00 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.299+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f63b8055050 con 0x56179d045e00 2024-04-24T22:52:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 10 client.16056 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56179d05fb00 con 0x56179d045e00 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b4005500 con 0x56179d045e00 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f63b8032440 con 0x56179d045e00 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 10 client.16056 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 20 client.16056 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d061080 0x56179d063470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 4 client.16056 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:52:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.300+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179cfb6ff0 con 0x56179d061080 2024-04-24T22:52:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.301+0000 7f63c6000640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d061080 0x56179d063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.301+0000 7f63c6000640 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d061080 0x56179d063470 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98534ee4a9a23ee server_cookie=b644718d35e7038e in_seq=0 out_seq=0 2024-04-24T22:52:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.301+0000 7f63a7e00640 10 client.16056 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63a7e00640 1 -- 172.21.15.111:0/848609190 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56179cfb6ff0 con 0x56179d061080 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63a7e00640 10 client.16056 handle_command_reply: tid=0 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 client.16056 shutdown 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 msgr2=0x56179d04df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f63b80057c0 tx=0x7f63b802a340 comp rx=0 tx=0).stop 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 shutdown_connections 2024-04-24T22:52:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d061080 0x56179d063470 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d04e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d04e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63a7e00640 0 client.16056 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 --2- 172.21.15.111:0/848609190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d04df60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.302+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 >> 172.21.15.111:0/848609190 conn(0x56179d042310 msgr2=0x56179d048090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.303+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 shutdown_connections 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.303+0000 7f63d0588740 1 -- 172.21.15.111:0/848609190 wait complete. 2024-04-24T22:52:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.303+0000 7f63d0588740 20 client.16056 trim_cache size 0 max 16384 2024-04-24T22:52:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 Processor -- start 2024-04-24T22:52:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 -- start start 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d05cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d0487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b1f30 con 0x56179d045e00 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81af800 con 0x56179d050fe0 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63d0588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ad670 con 0x56179d046720 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.319+0000 7f63c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47196/0 (socket says 172.21.15.111:47196) 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 -- 172.21.15.111:0/1823517834 learned_addr learned my addr 172.21.15.111:0/1823517834 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6000640 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d05cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c7400640 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d0487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 -- 172.21.15.111:0/1823517834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 msgr2=0x56179d05cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d05cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 -- 172.21.15.111:0/1823517834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 msgr2=0x56179d0487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d0487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 -- 172.21.15.111:0/1823517834 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179d04cf10 con 0x56179d045e00 2024-04-24T22:52:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.320+0000 7f63c6a00640 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f63b80095f0 tx=0x7f63b8009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83bbd96334b491d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63b2a00640 1 -- 172.21.15.111:0/1823517834 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b805e070 con 0x56179d045e00 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63b2a00640 1 -- 172.21.15.111:0/1823517834 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8032cd0 con 0x56179d045e00 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63b2a00640 1 -- 172.21.15.111:0/1823517834 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b804a050 con 0x56179d045e00 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 -- 172.21.15.111:0/1823517834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 msgr2=0x56179d050730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f63b80095f0 tx=0x7f63b8009450 comp rx=0 tx=0).stop 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 -- 172.21.15.111:0/1823517834 shutdown_connections 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d050fe0 0x56179d0487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d046720 0x56179d05cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 --2- 172.21.15.111:0/1823517834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d045e00 0x56179d050730 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 -- 172.21.15.111:0/1823517834 >> 172.21.15.111:0/1823517834 conn(0x56179d042310 msgr2=0x56179d050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 -- 172.21.15.111:0/1823517834 shutdown_connections 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.321+0000 7f63d0588740 1 -- 172.21.15.111:0/1823517834 wait complete. 2024-04-24T22:52:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.322+0000 7f63d0588740 1 -- start start 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.322+0000 7f63d0588740 1 Processor -- start 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d04adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d04b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81adcc0 con 0x56179d046720 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81af130 con 0x56179d045e00 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ae390 con 0x56179d050fe0 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63d0588740 10 client.0 _ll_register_callbacks cb 0x7f63b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47204/0 (socket says 172.21.15.111:47204) 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 -- 172.21.15.111:0/3623902749 learned_addr learned my addr 172.21.15.111:0/3623902749 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c7400640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d04b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6a00640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d04adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 -- 172.21.15.111:0/3623902749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 msgr2=0x56179d04b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d04b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 -- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 msgr2=0x56179d04adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d04adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.323+0000 7f63c6000640 1 -- 172.21.15.111:0/3623902749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179d0637e0 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63c6000640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f63b4011fd0 tx=0x7f63b401e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cad7ea81212e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b401ece0 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f63b402a410 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b8032cd0 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f63b4038050 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63d0588740 10 client.16062 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56179d0c78d0 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.324+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8032cd0 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f63b8030880 con 0x56179d046720 2024-04-24T22:52:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 20 client.16062 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d0c7d10 0x56179d0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 4 client.16062 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56179cfb1340 con 0x56179d0c7d10 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.325+0000 7f63c6a00640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d0c7d10 0x56179d0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.326+0000 7f63c6a00640 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d0c7d10 0x56179d0ca100 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc93581be8b1a14f server_cookie=ffe55633d36b5fa7 in_seq=0 out_seq=0 2024-04-24T22:52:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.326+0000 7f63a7e00640 10 client.16062 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.340+0000 7f63a7e00640 1 -- 172.21.15.111:0/3623902749 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56179cfb1340 con 0x56179d0c7d10 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.340+0000 7f63a7e00640 10 client.16062 handle_command_reply: tid=0 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.340+0000 7f63d0588740 1 client.16062 shutdown 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 msgr2=0x56179d04b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f63b4011fd0 tx=0x7f63b401e970 comp rx=0 tx=0).stop 2024-04-24T22:52:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 shutdown_connections 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x56179d0c7d10 0x56179d0ca100 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56179d050fe0 0x56179d04b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63a7e00640 0 client.16062 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56179d046720 0x56179d04b330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 --2- 172.21.15.111:0/3623902749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56179d045e00 0x56179d04adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 >> 172.21.15.111:0/3623902749 conn(0x56179d042310 msgr2=0x56179d05bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 shutdown_connections 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 1 -- 172.21.15.111:0/3623902749 wait complete. 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.341+0000 7f63d0588740 20 client.16062 trim_cache size 0 max 16384 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:52:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:52:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:52:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 msgr2=0x7f63ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 0x7f63ac0a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63b802a930 tx=0x7f63b80056d0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 msgr2=0x7f63c819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009fd0 tx=0x7f63c000c800 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 shutdown_connections 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63c8109930 0x7f63c819e9e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f63ac0a2da0 0x7f63ac0a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c81042e0 0x7f63c819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 --2- 172.21.15.111:0/1746412718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63c8103920 0x7f63c819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 >> 172.21.15.111:0/1746412718 conn(0x7f63c80ff430 msgr2=0x7f63c80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 shutdown_connections 2024-04-24T22:52:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:33.342+0000 7f63cd600640 1 -- 172.21.15.111:0/1746412718 wait complete. 2024-04-24T22:52:34.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:33 smithi186 ceph-mon[32753]: pgmap v331: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:52:34.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:33 smithi111 ceph-mon[27746]: pgmap v331: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:52:34.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:33 smithi171 ceph-mon[32337]: pgmap v331: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-24T22:52:36.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:35 smithi186 ceph-mon[32753]: pgmap v332: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:36.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:35 smithi111 ceph-mon[27746]: pgmap v332: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:36.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:35 smithi171 ceph-mon[32337]: pgmap v332: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:38.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:37 smithi186 ceph-mon[32753]: pgmap v333: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:38.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:37 smithi111 ceph-mon[27746]: pgmap v333: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:38.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:37 smithi171 ceph-mon[32337]: pgmap v333: 97 pgs: 97 active+clean; 5.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-24T22:52:40.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:39 smithi186 ceph-mon[32753]: pgmap v334: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-24T22:52:40.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:39 smithi111 ceph-mon[27746]: pgmap v334: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-24T22:52:40.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:39 smithi171 ceph-mon[32337]: pgmap v334: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-24T22:52:42.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:41 smithi111 ceph-mon[27746]: pgmap v335: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:52:42.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:41 smithi171 ceph-mon[32337]: pgmap v335: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:52:42.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:41 smithi186 ceph-mon[32753]: pgmap v335: 97 pgs: 97 active+clean; 1.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:52:44.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:44 smithi111 ceph-mon[27746]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:52:44.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:44 smithi171 ceph-mon[32337]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:52:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:44 smithi186 ceph-mon[32753]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-04-24T22:52:45.157 INFO:tasks.workunit.client.0.smithi111.stdout:499/600: open fd = 3 2024-04-24T22:52:45.157 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.157 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:500/600: open fd = 3 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:501/600: open fd = 3 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:502/600: open fd = 3 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:503/600: open fd = 3 2024-04-24T22:52:45.158 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:504/600: open fd = 3 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:505/600: open fd = 3 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:506/600: open fd = 3 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:507/600: open fd = 3 2024-04-24T22:52:45.159 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:508/600: open fd = 3 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:509/600: open fd = 3 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:510/600: open fd = 3 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:511/600: open fd = 3 2024-04-24T22:52:45.160 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:512/600: open fd = 3 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:513/600: open fd = 3 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:514/600: open fd = 3 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.161 INFO:tasks.workunit.client.0.smithi111.stdout:515/600: open fd = 3 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:516/600: open fd = 3 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:517/600: open fd = 3 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:518/600: open fd = 3 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.162 INFO:tasks.workunit.client.0.smithi111.stdout:519/600: open fd = 3 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:520/600: open fd = 3 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:521/600: open fd = 3 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.163 INFO:tasks.workunit.client.0.smithi111.stdout:522/600: open fd = 3 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:523/600: open fd = 3 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:524/600: open fd = 3 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:525/600: open fd = 3 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.164 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:526/600: open fd = 3 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:527/600: open fd = 3 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:528/600: open fd = 3 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:529/600: open fd = 3 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.165 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:530/600: open fd = 3 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:531/600: open fd = 3 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:532/600: open fd = 3 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:533/600: open fd = 3 2024-04-24T22:52:45.166 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:534/600: open fd = 3 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:535/600: open fd = 3 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:536/600: open fd = 3 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:537/600: open fd = 3 2024-04-24T22:52:45.167 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:538/600: open fd = 3 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:539/600: open fd = 3 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:540/600: open fd = 3 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.168 INFO:tasks.workunit.client.0.smithi111.stdout:541/600: open fd = 3 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:542/600: open fd = 3 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:543/600: open fd = 3 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:544/600: open fd = 3 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.169 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:545/600: open fd = 3 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:546/600: open fd = 3 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:547/600: open fd = 3 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.170 INFO:tasks.workunit.client.0.smithi111.stdout:548/600: open fd = 3 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:549/600: open fd = 3 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:550/600: open fd = 3 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:551/600: open fd = 3 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.171 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:552/600: open fd = 3 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:553/600: open fd = 3 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:554/600: open fd = 3 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:555/600: open fd = 3 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.172 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:556/600: open fd = 3 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:557/600: open fd = 3 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:558/600: open fd = 3 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:559/600: open fd = 3 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.173 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:45.174 INFO:tasks.workunit.client.0.smithi111.stdout:560/600: open fd = 3 2024-04-24T22:52:45.174 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.174 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:52:45.174 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:52:46.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:46 smithi111 ceph-mon[27746]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:46.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:46 smithi171 ceph-mon[32337]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 -- 172.21.15.111:0/3147944893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 msgr2=0x7f4b44109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3147944893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b44109b70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009930 tx=0x7f4b3402a420 comp rx=0 tx=0).stop 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 -- 172.21.15.111:0/3147944893 shutdown_connections 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3147944893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 0x7f4b44110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3147944893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b44103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3147944893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b44109b70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 -- 172.21.15.111:0/3147944893 >> 172.21.15.111:0/3147944893 conn(0x7f4b440ff250 msgr2=0x7f4b44101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 -- 172.21.15.111:0/3147944893 shutdown_connections 2024-04-24T22:52:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.483+0000 7f4b4a800640 1 -- 172.21.15.111:0/3147944893 wait complete. 2024-04-24T22:52:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.484+0000 7f4b4a800640 1 Processor -- start 2024-04-24T22:52:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.484+0000 7f4b4a800640 1 -- start start 2024-04-24T22:52:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.484+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.484+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 0x7f4b4419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.484+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b4419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b4a800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44113bf0 con 0x7f4b44109790 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b4a800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44113a70 con 0x7f4b44104140 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b4a800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44113d70 con 0x7f4b44103780 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:40062/0 (socket says 172.21.15.111:40062) 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 -- 172.21.15.111:0/3470899986 learned_addr learned my addr 172.21.15.111:0/3470899986 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49e00640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b4419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b48a00640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 0x7f4b4419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 -- 172.21.15.111:0/3470899986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 msgr2=0x7f4b4419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 0x7f4b4419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 -- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 msgr2=0x7f4b4419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b4419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 -- 172.21.15.111:0/3470899986 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4419f0b0 con 0x7f4b44103780 2024-04-24T22:52:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49e00640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b4419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.485+0000 7f4b49400640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402a940 tx=0x7f4b34003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.486+0000 7f4b42000640 1 -- 172.21.15.111:0/3470899986 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3403f070 con 0x7f4b44103780 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.486+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4419f340 con 0x7f4b44103780 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.486+0000 7f4b42000640 1 -- 172.21.15.111:0/3470899986 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34031c90 con 0x7f4b44103780 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.486+0000 7f4b42000640 1 -- 172.21.15.111:0/3470899986 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035030 con 0x7f4b44103780 2024-04-24T22:52:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.486+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a6c30 con 0x7f4b44103780 2024-04-24T22:52:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.489+0000 7f4b4ec84740 1 Processor -- start 2024-04-24T22:52:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.489+0000 7f4b42000640 1 -- 172.21.15.111:0/3470899986 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4b34042070 con 0x7f4b44103780 2024-04-24T22:52:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.489+0000 7f4b4ec84740 1 -- start start 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fcce34eab0 con 0x55fcce3f6190 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fcce2f7640 con 0x55fcce3eb8d0 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fcce34aa70 con 0x55fcce3eafb0 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b49e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34782/0 (socket says 172.21.15.111:34782) 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 -- 172.21.15.111:0/434872429 learned_addr learned my addr 172.21.15.111:0/434872429 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b49400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 -- 172.21.15.111:0/434872429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 msgr2=0x55fcce3eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 -- 172.21.15.111:0/434872429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 msgr2=0x55fcce3f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.490+0000 7f4b48a00640 1 -- 172.21.15.111:0/434872429 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce063210 con 0x55fcce3f6190 2024-04-24T22:52:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b42000640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b48a00640 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ca0 tx=0x7f4b38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab6f1a7768fddc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b42000640 1 -- 172.21.15.111:0/3470899986 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f4b340efb60 con 0x7f4b44103780 2024-04-24T22:52:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b32600640 1 -- 172.21.15.111:0/434872429 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3801e070 con 0x55fcce3f6190 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b49e00640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b32600640 1 -- 172.21.15.111:0/434872429 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38019b10 con 0x55fcce3f6190 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.491+0000 7f4b32600640 1 -- 172.21.15.111:0/434872429 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800ea80 con 0x55fcce3f6190 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 -- 172.21.15.111:0/434872429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 msgr2=0x55fcce3f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ca0 tx=0x7f4b38011810 comp rx=0 tx=0).stop 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b49e00640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c005fd0 tx=0x7f4b3c009500 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 -- 172.21.15.111:0/434872429 shutdown_connections 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce3f8580 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/434872429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 -- 172.21.15.111:0/434872429 >> 172.21.15.111:0/434872429 conn(0x55fcce3e74c0 msgr2=0x55fcce3e78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 -- 172.21.15.111:0/434872429 shutdown_connections 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.492+0000 7f4b4ec84740 1 -- 172.21.15.111:0/434872429 wait complete. 2024-04-24T22:52:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 -- start start 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 Processor -- start 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3f6190 0x55fcce3f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3f8dc0 con 0x55fcce3eb8d0 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fcce30a990 con 0x55fcce3f6190 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3f8f40 con 0x55fcce3eafb0 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.493+0000 7f4b4ec84740 10 client.0 _ll_register_callbacks cb 0x55fcce30c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34794/0 (socket says 172.21.15.111:34794) 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b48a00640 1 -- 172.21.15.111:0/2683940536 learned_addr learned my addr 172.21.15.111:0/2683940536 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49e00640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3f6190 0x55fcce3f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 -- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3f6190 msgr2=0x55fcce3f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3f6190 0x55fcce3f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 -- 172.21.15.111:0/2683940536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 msgr2=0x55fcce3f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 -- 172.21.15.111:0/2683940536 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce12f620 con 0x55fcce3eafb0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b48a00640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b49400640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3f3140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b340ed250 tx=0x7f4b3402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a327715ed86cd55c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.494+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34049030 con 0x55fcce3eafb0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f4b3403b460 con 0x55fcce3eafb0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b38019b10 con 0x55fcce3eafb0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b340bbb60 con 0x55fcce3eafb0 2024-04-24T22:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 10 client.26128 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fcce3f8e20 con 0x55fcce3eafb0 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38019b10 con 0x55fcce3eafb0 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b340b8690 con 0x55fcce3eafb0 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 10 client.26128 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 20 client.26128 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce406140 0x55fcce408570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 4 client.26128 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.495+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcce35bff0 con 0x55fcce406140 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.496+0000 7f4b48a00640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce406140 0x55fcce408570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.496+0000 7f4b48a00640 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce406140 0x55fcce408570 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141c9e497545b407 server_cookie=f3ca6cd20e36b6e8 in_seq=0 out_seq=0 2024-04-24T22:52:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.496+0000 7f4b26a00640 10 client.26128 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:46.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:46 smithi186 ceph-mon[32753]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.497+0000 7f4b26a00640 1 -- 172.21.15.111:0/2683940536 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcce35bff0 con 0x55fcce406140 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.497+0000 7f4b26a00640 10 client.26128 handle_command_reply: tid=0 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.497+0000 7f4b4ec84740 1 client.26128 shutdown 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 msgr2=0x55fcce3f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3f3140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b340ed250 tx=0x7f4b3402a420 comp rx=0 tx=0).stop 2024-04-24T22:52:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 shutdown_connections 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce406140 0x55fcce408570 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3f6190 0x55fcce3f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eb8d0 0x55fcce3f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/2683940536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eafb0 0x55fcce3f3140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 >> 172.21.15.111:0/2683940536 conn(0x55fcce3e74c0 msgr2=0x55fcce3ed1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b26a00640 0 client.26128 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 shutdown_connections 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 1 -- 172.21.15.111:0/2683940536 wait complete. 2024-04-24T22:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.498+0000 7f4b4ec84740 20 client.26128 trim_cache size 0 max 16384 2024-04-24T22:52:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.515+0000 7f4b4ec84740 1 Processor -- start 2024-04-24T22:52:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.515+0000 7f4b4ec84740 1 -- start start 2024-04-24T22:52:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.515+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eb8d0 0x55fcce3ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce40de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441b1d50 con 0x55fcce3f6190 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441af620 con 0x55fcce3eafb0 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441ad490 con 0x55fcce3eb8d0 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eb8d0 0x55fcce3ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce40de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:45134/0 (socket says 172.21.15.111:45134) 2024-04-24T22:52:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 -- 172.21.15.111:0/1895835287 learned_addr learned my addr 172.21.15.111:0/1895835287 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 -- 172.21.15.111:0/1895835287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eb8d0 msgr2=0x55fcce3ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eb8d0 0x55fcce3ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 -- 172.21.15.111:0/1895835287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 msgr2=0x55fcce40de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce40de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.516+0000 7f4b49400640 1 -- 172.21.15.111:0/1895835287 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce3f1690 con 0x55fcce3eafb0 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.517+0000 7f4b49400640 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b340b89f0 tx=0x7f4b340dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f66d37831f02c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.517+0000 7f4b31c00640 1 -- 172.21.15.111:0/1895835287 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34049070 con 0x55fcce3eafb0 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.517+0000 7f4b31c00640 1 -- 172.21.15.111:0/1895835287 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b340bc020 con 0x55fcce3eafb0 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.517+0000 7f4b31c00640 1 -- 172.21.15.111:0/1895835287 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b340bb980 con 0x55fcce3eafb0 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1895835287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 msgr2=0x55fcce401c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b340b89f0 tx=0x7f4b340dc040 comp rx=0 tx=0).stop 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1895835287 shutdown_connections 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3f6190 0x55fcce40de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3eb8d0 0x55fcce3ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1895835287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eafb0 0x55fcce401c80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1895835287 >> 172.21.15.111:0/1895835287 conn(0x55fcce3e74c0 msgr2=0x55fcce3ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1895835287 shutdown_connections 2024-04-24T22:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1895835287 wait complete. 2024-04-24T22:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 -- start start 2024-04-24T22:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.518+0000 7f4b4ec84740 1 Processor -- start 2024-04-24T22:52:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 0x55fcce3fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441adae0 con 0x55fcce3eafb0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441aef50 con 0x55fcce3eb8d0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441ae1b0 con 0x55fcce3f6190 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b4ec84740 10 client.0 _ll_register_callbacks cb 0x7f4b340f3300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b49400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b48a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b49400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34820/0 (socket says 172.21.15.111:34820) 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b49400640 1 -- 172.21.15.111:0/1278895928 learned_addr learned my addr 172.21.15.111:0/1278895928 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.519+0000 7f4b49e00640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 0x55fcce3fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 -- 172.21.15.111:0/1278895928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 msgr2=0x55fcce3fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 0x55fcce3fa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 -- 172.21.15.111:0/1278895928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 msgr2=0x55fcce3f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 -- 172.21.15.111:0/1278895928 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce473f00 con 0x55fcce3eb8d0 2024-04-24T22:52:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b49e00640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 0x55fcce3fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b49400640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b48a00640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3fa110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e740 tx=0x7f4b38009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0c6d67ed96a51e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3801e040 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcce356340 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f4b38004020 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b38039050 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b4ec84740 10 client.26090 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.520+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fcce4751f0 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fcce356340 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b3800e770 con 0x55fcce3eb8d0 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 10 client.26090 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 20 client.26090 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce475800 0x55fcce477c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 4 client.26090 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:52:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcce2d47a0 con 0x55fcce475800 2024-04-24T22:52:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.521+0000 7f4b49400640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce475800 0x55fcce477c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.522+0000 7f4b49400640 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce475800 0x55fcce477c20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631954c99bb74ca4 server_cookie=fdf932e3ef17388e in_seq=0 out_seq=0 2024-04-24T22:52:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.522+0000 7f4b26a00640 10 client.26090 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.536+0000 7f4b26a00640 1 -- 172.21.15.111:0/1278895928 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcce2d47a0 con 0x55fcce475800 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.536+0000 7f4b26a00640 10 client.26090 handle_command_reply: tid=0 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.536+0000 7f4b4ec84740 1 client.26090 shutdown 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.536+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 msgr2=0x55fcce3fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3fa110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e740 tx=0x7f4b38009bc0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 shutdown_connections 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55fcce475800 0x55fcce477c20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcce3f6190 0x55fcce3fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fcce3eb8d0 0x55fcce3fa110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b26a00640 0 client.26090 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 --2- 172.21.15.111:0/1278895928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcce3eafb0 0x55fcce3f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 >> 172.21.15.111:0/1278895928 conn(0x55fcce3e74c0 msgr2=0x55fcce3e5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 shutdown_connections 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 1 -- 172.21.15.111:0/1278895928 wait complete. 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.537+0000 7f4b4ec84740 20 client.26090 trim_cache size 0 max 16384 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:52:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 msgr2=0x7f4b2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c005fd0 tx=0x7f4b3c009500 comp rx=0 tx=0).stop 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 msgr2=0x7f4b4419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402a940 tx=0x7f4b34003b20 comp rx=0 tx=0).stop 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 shutdown_connections 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b44109790 0x7f4b4419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b44104140 0x7f4b4419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 --2- 172.21.15.111:0/3470899986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b44103780 0x7f4b4419a110 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 >> 172.21.15.111:0/3470899986 conn(0x7f4b440ff250 msgr2=0x7f4b440ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 shutdown_connections 2024-04-24T22:52:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:46.538+0000 7f4b4a800640 1 -- 172.21.15.111:0/3470899986 wait complete. 2024-04-24T22:52:48.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:48 smithi111 ceph-mon[27746]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:48.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:48 smithi171 ceph-mon[32337]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:48.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:48 smithi186 ceph-mon[32753]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-24T22:52:50.284 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:50 smithi111 ceph-mon[27746]: pgmap v339: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:52:50.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:50 smithi171 ceph-mon[32337]: pgmap v339: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:52:50.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:50 smithi186 ceph-mon[32753]: pgmap v339: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 20 op/s 2024-04-24T22:52:52.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:52 smithi111 ceph-mon[27746]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:52.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:52 smithi171 ceph-mon[32337]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:52.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:52 smithi186 ceph-mon[32753]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-24T22:52:54.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:54 smithi111 ceph-mon[27746]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:54.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:54 smithi171 ceph-mon[32337]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:54.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:54 smithi186 ceph-mon[32753]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-04-24T22:52:56.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:56 smithi171 ceph-mon[32337]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:56.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:56 smithi186 ceph-mon[32753]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:56.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:56 smithi111 ceph-mon[27746]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:58.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:52:58 smithi171 ceph-mon[32337]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:58.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:52:58 smithi186 ceph-mon[32753]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:58.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:52:58 smithi111 ceph-mon[27746]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-24T22:52:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 -- 172.21.15.111:0/350233581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e64075c70 msgr2=0x7f0e64076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/350233581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e64075c70 0x7f0e64076050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009ca0 tx=0x7f0e5402a8c0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 -- 172.21.15.111:0/350233581 shutdown_connections 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/350233581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e641125e0 0x7f0e641149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/350233581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64076590 0x7f0e641120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/350233581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e64075c70 0x7f0e64076050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 -- 172.21.15.111:0/350233581 >> 172.21.15.111:0/350233581 conn(0x7f0e640ff470 msgr2=0x7f0e64101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 -- 172.21.15.111:0/350233581 shutdown_connections 2024-04-24T22:52:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.680+0000 7f0e6ae00640 1 -- 172.21.15.111:0/350233581 wait complete. 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 Processor -- start 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 -- start start 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64075c70 0x7f0e6419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641181a0 con 0x7f0e641125e0 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64118020 con 0x7f0e64075c70 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.681+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64118320 con 0x7f0e64076590 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56000/0 (socket says 172.21.15.111:56000) 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56050/0 (socket says 172.21.15.111:56050) 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 -- 172.21.15.111:0/2751771238 learned_addr learned my addr 172.21.15.111:0/2751771238 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e63e00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64075c70 0x7f0e6419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 -- 172.21.15.111:0/2751771238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64075c70 msgr2=0x7f0e6419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64075c70 0x7f0e6419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 -- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 msgr2=0x7f0e641a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e5b400640 1 -- 172.21.15.111:0/2751771238 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e641a35b0 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.682+0000 7f0e68e00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9f0 tx=0x7f0e4c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e61600640 1 -- 172.21.15.111:0/2751771238 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00cd80 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e641a38a0 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e61600640 1 -- 172.21.15.111:0/2751771238 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4c010910 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e61600640 1 -- 172.21.15.111:0/2751771238 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0188f0 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.683+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e641ab130 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.686+0000 7f0e6db74740 1 Processor -- start 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.686+0000 7f0e61600640 1 -- 172.21.15.111:0/2751771238 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0e4c010ac0 con 0x7f0e64076590 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.686+0000 7f0e6db74740 1 -- start start 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c1dfe0 0x55f985c203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f985b76ab0 con 0x55f985c1dfe0 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f985b1f640 con 0x55f985c13720 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e6db74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f985b72a70 con 0x55f985c12e00 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56014/0 (socket says 172.21.15.111:56014) 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 -- 172.21.15.111:0/1459359626 learned_addr learned my addr 172.21.15.111:0/1459359626 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e5b400640 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c1dfe0 0x55f985c203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 -- 172.21.15.111:0/1459359626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 msgr2=0x55f985c1daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 -- 172.21.15.111:0/1459359626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c1dfe0 msgr2=0x55f985c203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c1dfe0 0x55f985c203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.687+0000 7f0e68e00640 1 -- 172.21.15.111:0/1459359626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98588b210 con 0x55f985c12e00 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e68e00640 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009f90 tx=0x7f0e5c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d32ad7a5f6c3483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e59600640 1 -- 172.21.15.111:0/1459359626 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c01e070 con 0x55f985c12e00 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e61600640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 0x7f0e480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e59600640 1 -- 172.21.15.111:0/1459359626 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c004310 con 0x55f985c12e00 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e59600640 1 -- 172.21.15.111:0/1459359626 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00f980 con 0x55f985c12e00 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e63e00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 0x7f0e480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.688+0000 7f0e61600640 1 -- 172.21.15.111:0/2751771238 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f0e4c0cf860 con 0x7f0e64076590 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 -- 172.21.15.111:0/1459359626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 msgr2=0x55f985c131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009f90 tx=0x7f0e5c0118d0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 -- 172.21.15.111:0/1459359626 shutdown_connections 2024-04-24T22:52:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c1dfe0 0x55f985c203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1459359626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c131e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 -- 172.21.15.111:0/1459359626 >> 172.21.15.111:0/1459359626 conn(0x55f985c0f310 msgr2=0x55f985c0f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e63e00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 0x7f0e480a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e54009820 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 -- 172.21.15.111:0/1459359626 shutdown_connections 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.689+0000 7f0e6db74740 1 -- 172.21.15.111:0/1459359626 wait complete. 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 -- start start 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 Processor -- start 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f985c20c10 con 0x55f985c12e00 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f985b32990 con 0x55f985c13720 2024-04-24T22:52:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f985c20d90 con 0x55f985c1dfe0 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.690+0000 7f0e6db74740 10 client.0 _ll_register_callbacks cb 0x55f985b34950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:36466/0 (socket says 172.21.15.111:36466) 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e68e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56018/0 (socket says 172.21.15.111:56018) 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c1afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56078/0 (socket says 172.21.15.111:56078) 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 -- 172.21.15.111:0/2438118122 learned_addr learned my addr 172.21.15.111:0/2438118122 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 -- 172.21.15.111:0/2438118122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 msgr2=0x55f985c1ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 -- 172.21.15.111:0/2438118122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 msgr2=0x55f985c1afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c1afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 -- 172.21.15.111:0/2438118122 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f985957620 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e68e00640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e5b400640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c09bd60 tx=0x7f0e4c0981a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fccb73fc2f2d91ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e52000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.691+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0cd3e0 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f0e4c09b420 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e5c004310 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c09bdd0 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 10 client.26102 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f985c2cb90 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c004310 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e4c097b20 con 0x55f985c13720 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 10 client.26102 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 20 client.26102 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c2e000 0x55f985c303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 4 client.26102 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:52:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.692+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f985b83ff0 con 0x55f985c2e000 2024-04-24T22:52:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.693+0000 7f0e63e00640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c2e000 0x55f985c303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.693+0000 7f0e63e00640 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c2e000 0x55f985c303f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94837376b217a948 server_cookie=6b42e1573aa65e6a in_seq=0 out_seq=0 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.693+0000 7f0e52000640 10 client.26102 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e52000640 1 -- 172.21.15.111:0/2438118122 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f985b83ff0 con 0x55f985c2e000 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e52000640 10 client.26102 handle_command_reply: tid=0 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e6db74740 1 client.26102 shutdown 2024-04-24T22:52:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 msgr2=0x55f985c1b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c09bd60 tx=0x7f0e4c0981a0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.694+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 shutdown_connections 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c2e000 0x55f985c303f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c1dfe0 0x55f985c1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c13720 0x55f985c1b500 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e52000640 0 client.26102 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 --2- 172.21.15.111:0/2438118122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c1afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 >> 172.21.15.111:0/2438118122 conn(0x55f985c0f310 msgr2=0x55f985c15040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 shutdown_connections 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 1 -- 172.21.15.111:0/2438118122 wait complete. 2024-04-24T22:52:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.695+0000 7f0e6db74740 20 client.26102 trim_cache size 0 max 16384 2024-04-24T22:52:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 Processor -- start 2024-04-24T22:52:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 -- start start 2024-04-24T22:52:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c13720 0x55f985c35d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b6340 con 0x55f985c12e00 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b3c10 con 0x55f985c1dfe0 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.711+0000 7f0e6db74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b1a80 con 0x55f985c13720 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56080/0 (socket says 172.21.15.111:56080) 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 -- 172.21.15.111:0/1314579302 learned_addr learned my addr 172.21.15.111:0/1314579302 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e68e00640 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 -- 172.21.15.111:0/1314579302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c13720 msgr2=0x55f985c35d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c13720 0x55f985c35d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 -- 172.21.15.111:0/1314579302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 msgr2=0x55f985c36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c36240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 -- 172.21.15.111:0/1314579302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f985c15db0 con 0x55f985c12e00 2024-04-24T22:52:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.712+0000 7f0e63e00640 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402f540 tx=0x7f0e54041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726c3df4f700056f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e58c00640 1 -- 172.21.15.111:0/1314579302 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54003cb0 con 0x55f985c12e00 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e58c00640 1 -- 172.21.15.111:0/1314579302 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e54059070 con 0x55f985c12e00 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e58c00640 1 -- 172.21.15.111:0/1314579302 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004040 con 0x55f985c12e00 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 -- 172.21.15.111:0/1314579302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 msgr2=0x55f985c29ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402f540 tx=0x7f0e54041000 comp rx=0 tx=0).stop 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 -- 172.21.15.111:0/1314579302 shutdown_connections 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c13720 0x55f985c35d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 --2- 172.21.15.111:0/1314579302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c12e00 0x55f985c29ad0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 -- 172.21.15.111:0/1314579302 >> 172.21.15.111:0/1314579302 conn(0x55f985c0f310 msgr2=0x55f985c09fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 -- 172.21.15.111:0/1314579302 shutdown_connections 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.713+0000 7f0e6db74740 1 -- 172.21.15.111:0/1314579302 wait complete. 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 -- start start 2024-04-24T22:52:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 Processor -- start 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c13720 0x55f985c18520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c18a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b20d0 con 0x55f985c13720 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b3540 con 0x55f985c1dfe0 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b27a0 con 0x55f985c12e00 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.714+0000 7f0e6db74740 10 client.0 _ll_register_callbacks cb 0x7f0e4c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:52:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:56046/0 (socket says 172.21.15.111:56046) 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 -- 172.21.15.111:0/483866912 learned_addr learned my addr 172.21.15.111:0/483866912 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e5b400640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c13720 0x55f985c18520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e68e00640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c18a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 -- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 msgr2=0x55f985c18a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c18a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 -- 172.21.15.111:0/483866912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c13720 msgr2=0x55f985c18520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c13720 0x55f985c18520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 -- 172.21.15.111:0/483866912 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f985c93840 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e63e00640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402a900 tx=0x7f0e540548b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702ffa8cfb164f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.715+0000 7f0e52000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54063070 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f0e54031450 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e54059070 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e5406f050 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 10 client.26149 fetch_fsmap learned FSMap version 24 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f985c94b50 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e54059070 con 0x55f985c12e00 2024-04-24T22:52:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e5402fd50 con 0x55f985c12e00 2024-04-24T22:52:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 10 client.26149 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:52:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 10 client.26149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.716+0000 7f0e6db74740 20 client.26149 populate_metadata read hostname 'smithi111' 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c952d0 0x55f985c976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e6db74740 4 client.26149 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f985b7e340 con 0x55f985c952d0 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e5b400640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c952d0 0x55f985c976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e52000640 10 client.26149 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.717+0000 7f0e5b400640 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c952d0 0x55f985c976f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6722608c67598026 server_cookie=d31fba50438675d2 in_seq=0 out_seq=0 2024-04-24T22:52:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e52000640 1 -- 172.21.15.111:0/483866912 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f985b7e340 con 0x55f985c952d0 2024-04-24T22:52:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e52000640 10 client.26149 handle_command_reply: tid=0 2024-04-24T22:52:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 client.26149 shutdown 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 msgr2=0x55f985c17fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402a900 tx=0x7f0e540548b0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 shutdown_connections 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f985c952d0 0x55f985c976f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f985c1dfe0 0x55f985c18a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f985c13720 0x55f985c18520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e52000640 0 client.26149 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 --2- 172.21.15.111:0/483866912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f985c12e00 0x55f985c17fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.733+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 >> 172.21.15.111:0/483866912 conn(0x55f985c0f310 msgr2=0x55f985c1af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 shutdown_connections 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6db74740 1 -- 172.21.15.111:0/483866912 wait complete. 2024-04-24T22:52:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6db74740 20 client.26149 trim_cache size 0 max 16384 2024-04-24T22:52:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:52:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:52:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:52:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 msgr2=0x7f0e480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 0x7f0e480a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e54009820 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 msgr2=0x7f0e6419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.734+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9f0 tx=0x7f0e4c00eeb0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 shutdown_connections 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e641125e0 0x7f0e641a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0e480a2da0 0x7f0e480a5260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e64076590 0x7f0e6419eb00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 --2- 172.21.15.111:0/2751771238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e64075c70 0x7f0e6419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 >> 172.21.15.111:0/2751771238 conn(0x7f0e640ff470 msgr2=0x7f0e64101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 shutdown_connections 2024-04-24T22:52:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:52:59.735+0000 7f0e6ae00640 1 -- 172.21.15.111:0/2751771238 wait complete. 2024-04-24T22:53:00.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:00 smithi171 ceph-mon[32337]: pgmap v344: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:53:00.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:00 smithi186 ceph-mon[32753]: pgmap v344: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:53:00.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:00 smithi111 ceph-mon[27746]: pgmap v344: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-24T22:53:02.336 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:02 smithi111 ceph-mon[27746]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:53:02.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:02 smithi171 ceph-mon[32337]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:53:02.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:02 smithi186 ceph-mon[32753]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-24T22:53:04.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:04 smithi171 ceph-mon[32337]: pgmap v346: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-04-24T22:53:04.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:04 smithi186 ceph-mon[32753]: pgmap v346: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-04-24T22:53:04.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:04 smithi111 ceph-mon[27746]: pgmap v346: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-04-24T22:53:06.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:06 smithi171 ceph-mon[32337]: pgmap v347: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 29 op/s 2024-04-24T22:53:06.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:06 smithi186 ceph-mon[32753]: pgmap v347: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 29 op/s 2024-04-24T22:53:06.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:06 smithi111 ceph-mon[27746]: pgmap v347: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 29 op/s 2024-04-24T22:53:07.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:07 smithi171 ceph-mon[32337]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T22:53:07.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:07 smithi186 ceph-mon[32753]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T22:53:07.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:07 smithi111 ceph-mon[27746]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T22:53:08.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:08 smithi171 ceph-mon[32337]: pgmap v349: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-04-24T22:53:08.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:08 smithi186 ceph-mon[32753]: pgmap v349: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-04-24T22:53:08.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:08 smithi111 ceph-mon[27746]: pgmap v349: 97 pgs: 97 active+clean; 2.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-04-24T22:53:10.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:10 smithi171 ceph-mon[32337]: pgmap v350: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:10.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:10 smithi186 ceph-mon[32753]: pgmap v350: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:10.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:10 smithi111 ceph-mon[27746]: pgmap v350: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:12.406 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:12 smithi171 ceph-mon[32337]: pgmap v351: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:12.499 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:12 smithi186 ceph-mon[32753]: pgmap v351: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:12.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:12 smithi111 ceph-mon[27746]: pgmap v351: 97 pgs: 97 active+clean; 3.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 -- 172.21.15.111:0/1244923973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 msgr2=0x7f8e181000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 --2- 172.21.15.111:0/1244923973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181000a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009900 tx=0x7f8e0002a3f0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 -- 172.21.15.111:0/1244923973 shutdown_connections 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 --2- 172.21.15.111:0/1244923973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e181006c0 0x7f8e18104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 --2- 172.21.15.111:0/1244923973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181000a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 --2- 172.21.15.111:0/1244923973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e18106880 0x7f8e18106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 -- 172.21.15.111:0/1244923973 >> 172.21.15.111:0/1244923973 conn(0x7f8e18076bc0 msgr2=0x7f8e180fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 -- 172.21.15.111:0/1244923973 shutdown_connections 2024-04-24T22:53:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.881+0000 7f8e1f000640 1 -- 172.21.15.111:0/1244923973 wait complete. 2024-04-24T22:53:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 Processor -- start 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 -- start start 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 0x7f8e181a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 0x7f8e1819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18118340 con 0x7f8e180ffc40 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181181c0 con 0x7f8e18106880 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.882+0000 7f8e1f000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181184c0 con 0x7f8e181006c0 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49106/0 (socket says 172.21.15.111:49106) 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 -- 172.21.15.111:0/2466409322 learned_addr learned my addr 172.21.15.111:0/2466409322 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e1d000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 0x7f8e1819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e0f400640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 0x7f8e181a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 -- 172.21.15.111:0/2466409322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 msgr2=0x7f8e181a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 0x7f8e181a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 -- 172.21.15.111:0/2466409322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 msgr2=0x7f8e1819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 0x7f8e1819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e17e00640 1 -- 172.21.15.111:0/2466409322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1819ed20 con 0x7f8e180ffc40 2024-04-24T22:53:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e1d000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 0x7f8e1819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.883+0000 7f8e0f400640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 0x7f8e181a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e17e00640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9f0 tx=0x7f8e0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e15600640 1 -- 172.21.15.111:0/2466409322 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800cd80 con 0x7f8e180ffc40 2024-04-24T22:53:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e15600640 1 -- 172.21.15.111:0/2466409322 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e08005500 con 0x7f8e180ffc40 2024-04-24T22:53:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1819f010 con 0x7f8e180ffc40 2024-04-24T22:53:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e15600640 1 -- 172.21.15.111:0/2466409322 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08010690 con 0x7f8e180ffc40 2024-04-24T22:53:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.884+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181ab2b0 con 0x7f8e180ffc40 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.887+0000 7f8e21f91740 1 Processor -- start 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.887+0000 7f8e15600640 1 -- 172.21.15.111:0/2466409322 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8e080046c0 con 0x7f8e180ffc40 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.887+0000 7f8e21f91740 1 -- start start 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d3e651ab0 con 0x559d3e6ede00 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559d3e5fa640 con 0x559d3e6f8fe0 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e21f91740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559d3e64da70 con 0x559d3e6ee720 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49122/0 (socket says 172.21.15.111:49122) 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e1d000640 1 -- 172.21.15.111:0/3492827578 learned_addr learned my addr 172.21.15.111:0/3492827578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e0f400640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.888+0000 7f8e17e00640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 -- 172.21.15.111:0/3492827578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 msgr2=0x559d3e6f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 -- 172.21.15.111:0/3492827578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 msgr2=0x559d3e6fb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e15600640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 0x7f8dfc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6fb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 -- 172.21.15.111:0/3492827578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3e366210 con 0x559d3e6ede00 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e0f400640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e15600640 1 -- 172.21.15.111:0/2466409322 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f8e080cf7f0 con 0x7f8e180ffc40 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 0x7f8dfc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.889+0000 7f8e1d000640 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8e100107e0 tx=0x7f8e10010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2b96653e907adf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.890+0000 7f8e0d600640 1 -- 172.21.15.111:0/3492827578 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10022070 con 0x559d3e6ede00 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.890+0000 7f8e0d600640 1 -- 172.21.15.111:0/3492827578 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e1000ed70 con 0x559d3e6ede00 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.890+0000 7f8e0d600640 1 -- 172.21.15.111:0/3492827578 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1001b760 con 0x559d3e6ede00 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- 172.21.15.111:0/3492827578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 msgr2=0x559d3e6ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8e100107e0 tx=0x7f8e10010ca0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e1d000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 0x7f8dfc0a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e1001f000 tx=0x7f8e10020000 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- 172.21.15.111:0/3492827578 shutdown_connections 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 --2- 172.21.15.111:0/3492827578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6ee1e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- 172.21.15.111:0/3492827578 >> 172.21.15.111:0/3492827578 conn(0x559d3e6ea310 msgr2=0x559d3e6ea6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- 172.21.15.111:0/3492827578 shutdown_connections 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- 172.21.15.111:0/3492827578 wait complete. 2024-04-24T22:53:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 -- start start 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.891+0000 7f8e21f91740 1 Processor -- start 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d3e6fbc10 con 0x559d3e6ede00 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559d3e60d990 con 0x559d3e6f8fe0 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559d3e6fbd90 con 0x559d3e6ee720 2024-04-24T22:53:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e21f91740 10 client.0 _ll_register_callbacks cb 0x559d3e60f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49132/0 (socket says 172.21.15.111:49132) 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e17e00640 1 -- 172.21.15.111:0/4155760762 learned_addr learned my addr 172.21.15.111:0/4155760762 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e0f400640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.892+0000 7f8e1d000640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 -- 172.21.15.111:0/4155760762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 msgr2=0x559d3e6f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 -- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 msgr2=0x559d3e6f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 -- 172.21.15.111:0/4155760762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d3e432620 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e0f400640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e17e00640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8e0809ba10 tx=0x7f8e08004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74477b5430c7a372 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e06000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0809b670 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f8e080a46f0 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1000ed70 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8e080cd3d0 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e21f91740 10 client.16110 fetch_fsmap learned FSMap version 24 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.893+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559d3e707b60 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e1000ed70 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8e0809b1f0 con 0x559d3e6ede00 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 10 client.16110 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:53:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 20 client.16110 populate_metadata read hostname 'smithi111' 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e709070 0x559d3e70b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 4 client.16110 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d3e65eff0 con 0x559d3e709070 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.894+0000 7f8e0f400640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e709070 0x559d3e70b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.895+0000 7f8e0f400640 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e709070 0x559d3e70b460 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176610467dd52188 server_cookie=e9bba4a895839418 in_seq=0 out_seq=0 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.895+0000 7f8e06000640 10 client.16110 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e06000640 1 -- 172.21.15.111:0/4155760762 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d3e65eff0 con 0x559d3e709070 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e06000640 10 client.16110 handle_command_reply: tid=0 2024-04-24T22:53:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 client.16110 shutdown 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 msgr2=0x559d3e6f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8e0809ba10 tx=0x7f8e08004ae0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 shutdown_connections 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e709070 0x559d3e70b460 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ee720 0x559d3e6f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e06000640 0 client.16110 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 --2- 172.21.15.111:0/4155760762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ede00 0x559d3e6f5f80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 >> 172.21.15.111:0/4155760762 conn(0x559d3e6ea310 msgr2=0x559d3e6f0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 shutdown_connections 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 1 -- 172.21.15.111:0/4155760762 wait complete. 2024-04-24T22:53:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.896+0000 7f8e21f91740 20 client.16110 trim_cache size 0 max 16384 2024-04-24T22:53:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 Processor -- start 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 -- start start 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 0x559d3e704ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b6300 con 0x559d3e6ee720 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b3bd0 con 0x559d3e6ede00 2024-04-24T22:53:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e21f91740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b1a40 con 0x559d3e6f8fe0 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e0f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e0f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49136/0 (socket says 172.21.15.111:49136) 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e0f400640 1 -- 172.21.15.111:0/54280133 learned_addr learned my addr 172.21.15.111:0/54280133 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e17e00640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 0x559d3e704ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.913+0000 7f8e1d000640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 -- 172.21.15.111:0/54280133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6f8fe0 msgr2=0x559d3e6f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 -- 172.21.15.111:0/54280133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 msgr2=0x559d3e704ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 0x559d3e704ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 -- 172.21.15.111:0/54280133 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3e70b7d0 con 0x559d3e6ee720 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e17e00640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 0x559d3e704ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0f400640 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009900 tx=0x7f8e00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c16cc570a4aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0cc00640 1 -- 172.21.15.111:0/54280133 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00035040 con 0x559d3e6ee720 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0cc00640 1 -- 172.21.15.111:0/54280133 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e00047910 con 0x559d3e6ee720 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.914+0000 7f8e0cc00640 1 -- 172.21.15.111:0/54280133 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000317b0 con 0x559d3e6ee720 2024-04-24T22:53:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 -- 172.21.15.111:0/54280133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 msgr2=0x559d3e6f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009900 tx=0x7f8e00009260 comp rx=0 tx=0).stop 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 -- 172.21.15.111:0/54280133 shutdown_connections 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6f8fe0 0x559d3e6f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6ee720 0x559d3e6f6820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 --2- 172.21.15.111:0/54280133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ede00 0x559d3e704ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 -- 172.21.15.111:0/54280133 >> 172.21.15.111:0/54280133 conn(0x559d3e6ea310 msgr2=0x559d3e6f77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 -- 172.21.15.111:0/54280133 shutdown_connections 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.915+0000 7f8e21f91740 1 -- 172.21.15.111:0/54280133 wait complete. 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 -- start start 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 Processor -- start 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 0x559d3e6f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ee720 0x559d3e6f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b2090 con 0x559d3e6f8fe0 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b3500 con 0x559d3e6ee720 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181b2760 con 0x559d3e6ede00 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.916+0000 7f8e21f91740 10 client.0 _ll_register_callbacks cb 0x7f8e080d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49142/0 (socket says 172.21.15.111:49142) 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e1d000640 1 -- 172.21.15.111:0/1253714612 learned_addr learned my addr 172.21.15.111:0/1253714612 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 0x559d3e6f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e0f400640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ee720 0x559d3e6f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 -- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ee720 msgr2=0x559d3e6f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ee720 0x559d3e6f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 -- 172.21.15.111:0/1253714612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 msgr2=0x559d3e76e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 -- 172.21.15.111:0/1253714612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d3e6f3db0 con 0x559d3e6ede00 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e1d000640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e17e00640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 0x559d3e6f2fa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e080986e0 tx=0x7f8e0809abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aec32464601098f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e06000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:53:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.917+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080d5040 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e00047910 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f8e08097a60 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8e0809b670 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 10 client.26167 fetch_fsmap learned FSMap version 24 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559d3e76fbb0 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e00047910 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8e080a4bd0 con 0x559d3e6ede00 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 10 client.26167 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 20 client.26167 populate_metadata read hostname 'smithi111' 2024-04-24T22:53:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e7702d0 0x559d3e7726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 4 client.26167 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:53:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.918+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d3e659340 con 0x559d3e7702d0 2024-04-24T22:53:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.919+0000 7f8e0f400640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e7702d0 0x559d3e7726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.919+0000 7f8e0f400640 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e7702d0 0x559d3e7726c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ca327aa64a0b7e server_cookie=cdd67abfc0d2dd7b in_seq=0 out_seq=0 2024-04-24T22:53:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.919+0000 7f8e06000640 10 client.26167 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e06000640 1 -- 172.21.15.111:0/1253714612 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d3e659340 con 0x559d3e7702d0 2024-04-24T22:53:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e06000640 10 client.26167 handle_command_reply: tid=0 2024-04-24T22:53:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 client.26167 shutdown 2024-04-24T22:53:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 msgr2=0x559d3e6f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 0x559d3e6f2fa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e080986e0 tx=0x7f8e0809abd0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 shutdown_connections 2024-04-24T22:53:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x559d3e7702d0 0x559d3e7726c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d3e6f8fe0 0x559d3e76e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559d3e6ee720 0x559d3e6f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 --2- 172.21.15.111:0/1253714612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3e6ede00 0x559d3e6f2fa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e06000640 0 client.26167 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 >> 172.21.15.111:0/1253714612 conn(0x559d3e6ea310 msgr2=0x559d3e6f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 shutdown_connections 2024-04-24T22:53:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 1 -- 172.21.15.111:0/1253714612 wait complete. 2024-04-24T22:53:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.934+0000 7f8e21f91740 20 client.26167 trim_cache size 0 max 16384 2024-04-24T22:53:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:53:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:53:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:53:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:53:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 msgr2=0x7f8dfc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 0x7f8dfc0a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e1001f000 tx=0x7f8e10020000 comp rx=0 tx=0).stop 2024-04-24T22:53:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 msgr2=0x7f8e181a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9f0 tx=0x7f8e0800edf0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 shutdown_connections 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e18106880 0x7f8e1819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8dfc0a2da0 0x7f8dfc0a5260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e181006c0 0x7f8e181a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 --2- 172.21.15.111:0/2466409322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e180ffc40 0x7f8e181a43f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.935+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 >> 172.21.15.111:0/2466409322 conn(0x7f8e18076bc0 msgr2=0x7f8e18101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.936+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 shutdown_connections 2024-04-24T22:53:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:12.936+0000 7f8e1f000640 1 -- 172.21.15.111:0/2466409322 wait complete. 2024-04-24T22:53:13.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:13 smithi111 ceph-mon[27746]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T22:53:13.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:13 smithi171 ceph-mon[32337]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T22:53:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:13 smithi186 ceph-mon[32753]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T22:53:14.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:14 smithi111 ceph-mon[27746]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-04-24T22:53:14.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:14 smithi171 ceph-mon[32337]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-04-24T22:53:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:14 smithi186 ceph-mon[32753]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-04-24T22:53:16.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:16 smithi111 ceph-mon[27746]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-04-24T22:53:16.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:16 smithi171 ceph-mon[32337]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-04-24T22:53:16.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:16 smithi186 ceph-mon[32753]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-04-24T22:53:18.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:18 smithi111 ceph-mon[27746]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-04-24T22:53:18.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:18 smithi171 ceph-mon[32337]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-04-24T22:53:18.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:18 smithi186 ceph-mon[32753]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-04-24T22:53:20.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:20 smithi111 ceph-mon[27746]: pgmap v356: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:20.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:20 smithi171 ceph-mon[32337]: pgmap v356: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:20.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:20 smithi186 ceph-mon[32753]: pgmap v356: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:22.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:22 smithi111 ceph-mon[27746]: pgmap v357: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:22.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:22 smithi171 ceph-mon[32337]: pgmap v357: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:22.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:22 smithi186 ceph-mon[32753]: pgmap v357: 97 pgs: 97 active+clean; 3.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-04-24T22:53:24.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:24 smithi111 ceph-mon[27746]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:53:24.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:24 smithi171 ceph-mon[32337]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:53:24.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:24 smithi186 ceph-mon[32753]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-04-24T22:53:25.736 INFO:tasks.workunit.client.0.smithi111.stdout:561/600: open fd = 3 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:562/600: open fd = 3 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:563/600: open fd = 3 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:564/600: open fd = 3 2024-04-24T22:53:25.737 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:565/600: open fd = 3 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:566/600: open fd = 3 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:567/600: open fd = 3 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.738 INFO:tasks.workunit.client.0.smithi111.stdout:568/600: open fd = 3 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:569/600: open fd = 3 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:570/600: open fd = 3 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:571/600: open fd = 3 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.739 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:572/600: open fd = 3 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:573/600: open fd = 3 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:574/600: open fd = 3 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:575/600: open fd = 3 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.740 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:576/600: open fd = 3 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:577/600: open fd = 3 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.741 INFO:tasks.workunit.client.0.smithi111.stdout:578/600: open fd = 3 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:579/600: open fd = 3 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:580/600: open fd = 3 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:581/600: open fd = 3 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.742 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:582/600: open fd = 3 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:583/600: open fd = 3 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:584/600: open fd = 3 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:585/600: open fd = 3 2024-04-24T22:53:25.743 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:586/600: open fd = 3 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:587/600: open fd = 3 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:588/600: open fd = 3 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.744 INFO:tasks.workunit.client.0.smithi111.stdout:589/600: open fd = 3 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:590/600: open fd = 3 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:591/600: open fd = 3 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:592/600: open fd = 3 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.745 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:593/600: open fd = 3 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:594/600: open fd = 3 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.746 INFO:tasks.workunit.client.0.smithi111.stdout:595/600: open fd = 3 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:596/600: open fd = 3 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:597/600: open fd = 3 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:598/600: open fd = 3 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.747 INFO:tasks.workunit.client.0.smithi111.stdout:599/600: open fd = 3 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:600/600: open fd = 3 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:write ret = 32 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stdout:pread ret = 64 2024-04-24T22:53:25.748 INFO:tasks.workunit.client.0.smithi111.stderr:+ echo OK 2024-04-24T22:53:25.749 INFO:tasks.workunit.client.0.smithi111.stdout:OK 2024-04-24T22:53:25.750 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T22:53:25.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T22:53:25.894 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-24T22:53:25.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T22:53:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 -- 172.21.15.111:0/138390784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04102110 msgr2=0x7f2d0410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 --2- 172.21.15.111:0/138390784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04102110 0x7f2d0410e620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009be0 tx=0x7f2cfc02a6c0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 -- 172.21.15.111:0/138390784 shutdown_connections 2024-04-24T22:53:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 --2- 172.21.15.111:0/138390784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04102110 0x7f2d0410e620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 --2- 172.21.15.111:0/138390784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04101750 0x7f2d04101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 --2- 172.21.15.111:0/138390784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04107760 0x7f2d04107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 -- 172.21.15.111:0/138390784 >> 172.21.15.111:0/138390784 conn(0x7f2d04078bd0 msgr2=0x7f2d04078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 -- 172.21.15.111:0/138390784 shutdown_connections 2024-04-24T22:53:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.070+0000 7f2d09e00640 1 -- 172.21.15.111:0/138390784 wait complete. 2024-04-24T22:53:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.072+0000 7f2d09e00640 1 Processor -- start 2024-04-24T22:53:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.072+0000 7f2d09e00640 1 -- start start 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.072+0000 7f2d09e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.072+0000 7f2d09e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04102110 0x7f2d04198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.072+0000 7f2d09e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04107760 0x7f2d0419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d09e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111b70 con 0x7f2d04107760 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d09e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041119f0 con 0x7f2d04101750 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d09e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111cf0 con 0x7f2d04102110 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04102110 0x7f2d04198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04107760 0x7f2d0419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52646/0 (socket says 172.21.15.111:52646) 2024-04-24T22:53:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 -- 172.21.15.111:0/381259800 learned_addr learned my addr 172.21.15.111:0/381259800 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 -- 172.21.15.111:0/381259800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04102110 msgr2=0x7f2d04198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04102110 0x7f2d04198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 -- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04107760 msgr2=0x7f2d0419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04107760 0x7f2d0419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.073+0000 7f2d02a00640 1 -- 172.21.15.111:0/381259800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0419ceb0 con 0x7f2d04101750 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d02a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9c0 tx=0x7f2cf400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d08a00640 1 -- 172.21.15.111:0/381259800 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400ccc0 con 0x7f2d04101750 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d08a00640 1 -- 172.21.15.111:0/381259800 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cf4005500 con 0x7f2d04101750 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d08a00640 1 -- 172.21.15.111:0/381259800 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4010430 con 0x7f2d04101750 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0419d1a0 con 0x7f2d04101750 2024-04-24T22:53:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.074+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d04073970 con 0x7f2d04101750 2024-04-24T22:53:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.077+0000 7f2d0cb82740 1 Processor -- start 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 -- start start 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 0x55f312d5ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f312cacab0 con 0x55f312d5c7f0 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f312c55640 con 0x55f312d48d40 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f312ca8a70 con 0x55f312d49660 2024-04-24T22:53:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52660/0 (socket says 172.21.15.111:52660) 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d03400640 1 -- 172.21.15.111:0/4043006524 learned_addr learned my addr 172.21.15.111:0/4043006524 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d02000640 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 0x55f312d5ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.078+0000 7f2d08a00640 1 -- 172.21.15.111:0/381259800 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2cf40046c0 con 0x7f2d04101750 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 -- 172.21.15.111:0/4043006524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 msgr2=0x55f312d5c2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d5c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 -- 172.21.15.111:0/4043006524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 msgr2=0x55f312d5ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 0x55f312d5ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 -- 172.21.15.111:0/4043006524 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3129c1210 con 0x55f312d48d40 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d02000640 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 0x55f312d5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2d03400640 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009be0 tx=0x7f2cfc032e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4110cbf267621b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2cefe00640 1 -- 172.21.15.111:0/4043006524 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc041070 con 0x55f312d48d40 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2cefe00640 1 -- 172.21.15.111:0/4043006524 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc0305a0 con 0x55f312d48d40 2024-04-24T22:53:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.079+0000 7f2cefe00640 1 -- 172.21.15.111:0/4043006524 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc036040 con 0x55f312d48d40 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.080+0000 7f2d08a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 0x7f2ce80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.080+0000 7f2d08a00640 1 -- 172.21.15.111:0/381259800 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f2cf40cf670 con 0x7f2d04101750 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.080+0000 7f2d0cb82740 1 -- 172.21.15.111:0/4043006524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 msgr2=0x55f312d49120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.080+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009be0 tx=0x7f2cfc032e10 comp rx=0 tx=0).stop 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d02a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 0x7f2ce80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 -- 172.21.15.111:0/4043006524 shutdown_connections 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d5c7f0 0x55f312d5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d5c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/4043006524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d49120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 -- 172.21.15.111:0/4043006524 >> 172.21.15.111:0/4043006524 conn(0x55f312d45310 msgr2=0x55f312d456f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 -- 172.21.15.111:0/4043006524 shutdown_connections 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d0cb82740 1 -- 172.21.15.111:0/4043006524 wait complete. 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.081+0000 7f2d02a00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 0x7f2ce80a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40a4270 tx=0x7f2cf400ce60 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.082+0000 7f2d0cb82740 1 -- start start 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.082+0000 7f2d0cb82740 1 Processor -- start 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 0x55f312d59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d5c7f0 0x55f312d5a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f312d5f360 con 0x55f312d49660 2024-04-24T22:53:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f312c68990 con 0x55f312d48d40 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f312d5f4e0 con 0x55f312d5c7f0 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d0cb82740 10 client.0 _ll_register_callbacks cb 0x55f312c6a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d02000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 0x55f312d59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d59750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52676/0 (socket says 172.21.15.111:52676) 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d03400640 1 -- 172.21.15.111:0/2011817180 learned_addr learned my addr 172.21.15.111:0/2011817180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d03400640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d5c7f0 0x55f312d5a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d02000640 1 -- 172.21.15.111:0/2011817180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d5c7f0 msgr2=0x55f312d5a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.083+0000 7f2d02000640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d5c7f0 0x55f312d5a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2d02000640 1 -- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 msgr2=0x55f312d59750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2d02000640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d59750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2d02000640 1 -- 172.21.15.111:0/2011817180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f312a8d620 con 0x55f312d49660 2024-04-24T22:53:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2d02000640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 0x55f312d59c90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55f312d4b920 tx=0x7f2cf0009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a4aac15afddb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2cecc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf001e020 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f2cf0004460 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.084+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc0305a0 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2cf0009e30 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 10 client.16128 fetch_fsmap learned FSMap version 24 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f312d6b350 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc0305a0 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2cf0019070 con 0x55f312d49660 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 10 client.16128 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:53:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 20 client.16128 populate_metadata read hostname 'smithi111' 2024-04-24T22:53:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312d6c810 0x55f312d6ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 4 client.16128 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:53:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.085+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f312cb9ff0 con 0x55f312d6c810 2024-04-24T22:53:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.086+0000 7f2d03400640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312d6c810 0x55f312d6ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.086+0000 7f2d03400640 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312d6c810 0x55f312d6ec20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3460d92eb3228d9d server_cookie=a349d13879691884 in_seq=0 out_seq=0 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.086+0000 7f2cecc00640 10 client.16128 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.087+0000 7f2cecc00640 1 -- 172.21.15.111:0/2011817180 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f312cb9ff0 con 0x55f312d6c810 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.087+0000 7f2cecc00640 10 client.16128 handle_command_reply: tid=0 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.087+0000 7f2d0cb82740 1 client.16128 shutdown 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.087+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 msgr2=0x55f312d59c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.087+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 0x55f312d59c90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55f312d4b920 tx=0x7f2cf0009c40 comp rx=0 tx=0).stop 2024-04-24T22:53:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 shutdown_connections 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312d6c810 0x55f312d6ec20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d5c7f0 0x55f312d5a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2cecc00640 0 client.16128 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d49660 0x55f312d59c90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/2011817180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d48d40 0x55f312d59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 >> 172.21.15.111:0/2011817180 conn(0x55f312d45310 msgr2=0x55f312d4b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 shutdown_connections 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 1 -- 172.21.15.111:0/2011817180 wait complete. 2024-04-24T22:53:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.088+0000 7f2d0cb82740 20 client.16128 trim_cache size 0 max 16384 2024-04-24T22:53:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.119+0000 7f2d0cb82740 1 Processor -- start 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.119+0000 7f2d0cb82740 1 -- start start 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d68290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d74530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ad9d0 con 0x55f312d48d40 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ab2a0 con 0x55f312d5c7f0 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041a9110 con 0x55f312d49660 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d68290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d02000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d68290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51382/0 (socket says 172.21.15.111:51382) 2024-04-24T22:53:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d02000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53108/0 (socket says 172.21.15.111:53108) 2024-04-24T22:53:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d03400640 1 -- 172.21.15.111:0/337849267 learned_addr learned my addr 172.21.15.111:0/337849267 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d03400640 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d74530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.120+0000 7f2d02000640 1 -- 172.21.15.111:0/337849267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 msgr2=0x55f312d74530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d02000640 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d74530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d02000640 1 -- 172.21.15.111:0/337849267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 msgr2=0x55f312d68290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d02000640 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d68290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d02000640 1 -- 172.21.15.111:0/337849267 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f312d6f2a0 con 0x55f312d49660 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d03400640 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2d02000640 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0003ec0 tx=0x7f2cf0011fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332aaea321b49be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2cef400640 1 -- 172.21.15.111:0/337849267 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf001e070 con 0x55f312d49660 2024-04-24T22:53:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2cef400640 1 -- 172.21.15.111:0/337849267 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cf000fd50 con 0x55f312d49660 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.121+0000 7f2cef400640 1 -- 172.21.15.111:0/337849267 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0005620 con 0x55f312d49660 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.122+0000 7f2d0cb82740 1 -- 172.21.15.111:0/337849267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 msgr2=0x55f312d59e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.122+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0003ec0 tx=0x7f2cf0011fc0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 -- 172.21.15.111:0/337849267 shutdown_connections 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d59e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/337849267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d68290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 -- 172.21.15.111:0/337849267 >> 172.21.15.111:0/337849267 conn(0x55f312d45310 msgr2=0x55f312d59710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 -- 172.21.15.111:0/337849267 shutdown_connections 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 -- 172.21.15.111:0/337849267 wait complete. 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.123+0000 7f2d0cb82740 1 -- start start 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 Processor -- start 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d64bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041a9760 con 0x55f312d48d40 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041aabd0 con 0x55f312d5c7f0 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041a9e30 con 0x55f312d49660 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.124+0000 7f2d0cb82740 10 client.0 _ll_register_callbacks cb 0x7f2cf0010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:53:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d64bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53114/0 (socket says 172.21.15.111:53114) 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 -- 172.21.15.111:0/3198216234 learned_addr learned my addr 172.21.15.111:0/3198216234 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d03400640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 -- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 msgr2=0x55f312d61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 -- 172.21.15.111:0/3198216234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 msgr2=0x55f312d64bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d64bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d02000640 1 -- 172.21.15.111:0/3198216234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f312d62500 con 0x55f312d49660 2024-04-24T22:53:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.125+0000 7f2d03400640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d64bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.126+0000 7f2d02000640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009780 tx=0x7f2cf0009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651c2bc5a48c678c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.126+0000 7f2cecc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.126+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf001e040 con 0x55f312d49660 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.126+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f2cf0005520 con 0x55f312d49660 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.126+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf000fd50 con 0x55f312d49660 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2cf0005b90 con 0x55f312d49660 2024-04-24T22:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2d0cb82740 10 client.26182 fetch_fsmap learned FSMap version 24 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f312dd3e50 con 0x55f312d49660 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cf000fd50 con 0x55f312d49660 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2cf003b020 con 0x55f312d49660 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2d0cb82740 10 client.26182 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2d0cb82740 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.127+0000 7f2d0cb82740 20 client.26182 populate_metadata read hostname 'smithi111' 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312dd4500 0x55f312dd6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2d0cb82740 4 client.26182 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 --> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f312cb4340 con 0x55f312dd4500 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2d03400640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312dd4500 0x55f312dd6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2cecc00640 10 client.26182 ms_handle_connect on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.128+0000 7f2d03400640 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312dd4500 0x55f312dd6920 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584236f36545265 server_cookie=e5a1a3b61783324a in_seq=0 out_seq=0 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2cecc00640 1 -- 172.21.15.111:0/3198216234 <== mds.0 v2:172.21.15.171:6838/3821671666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f312cb4340 con 0x55f312dd4500 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2cecc00640 10 client.26182 handle_command_reply: tid=0 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 client.26182 shutdown 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 msgr2=0x55f312d61770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009780 tx=0x7f2cf0009c40 comp rx=0 tx=0).stop 2024-04-24T22:53:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 shutdown_connections 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:6838/3821671666,v1:172.21.15.171:6839/3821671666] conn(0x55f312dd4500 0x55f312dd6920 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f312d5c7f0 0x55f312d61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f312d49660 0x55f312d61770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 --2- 172.21.15.111:0/3198216234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f312d48d40 0x55f312d64bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 >> 172.21.15.111:0/3198216234 conn(0x55f312d45310 msgr2=0x55f312d4b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2cecc00640 0 client.26182 ms_handle_reset on v2:172.21.15.171:6838/3821671666 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 shutdown_connections 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.150+0000 7f2d0cb82740 1 -- 172.21.15.111:0/3198216234 wait complete. 2024-04-24T22:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d0cb82740 20 client.26182 trim_cache size 0 max 16384 2024-04-24T22:53:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-04-24T22:53:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-04-24T22:53:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-04-24T22:53:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 msgr2=0x7f2ce80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 0x7f2ce80a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40a4270 tx=0x7f2cf400ce60 comp rx=0 tx=0).stop 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 msgr2=0x7f2d04197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9c0 tx=0x7f2cf400ee80 comp rx=0 tx=0).stop 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 shutdown_connections 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d04107760 0x7f2d0419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f2ce80a2e20 0x7f2ce80a52e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d04102110 0x7f2d04198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 --2- 172.21.15.111:0/381259800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d04101750 0x7f2d04197f10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.151+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 >> 172.21.15.111:0/381259800 conn(0x7f2d04078bd0 msgr2=0x7f2d040fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.152+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 shutdown_connections 2024-04-24T22:53:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-04-24T22:53:26.152+0000 7f2d09e00640 1 -- 172.21.15.111:0/381259800 wait complete. 2024-04-24T22:53:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:26 smithi171 ceph-mon[32337]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:26 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:26 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:26.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:26 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:26 smithi186 ceph-mon[32753]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:26 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:26 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:26.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:26 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:26 smithi111 ceph-mon[27746]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:26 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:26 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:26.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:26 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:27.059 DEBUG:teuthology.parallel:result is None 2024-04-24T22:53:27.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:53:27.185 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:53:27.185 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-24T22:53:27.198 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:53:27.223 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-24T22:53:27.233 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-24T22:53:27.233 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T22:53:27.234 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:53:27.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-24T22:53:27.336 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:27 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:27.336 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:27 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:27.336 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:27 smithi111 ceph-mon[27746]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:27.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-24T22:53:27.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T22:53:27.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 -- 172.21.15.111:0/1106938683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 msgr2=0x7fec2810dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:27.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 --2- 172.21.15.111:0/1106938683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec2810dff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec18009860 tx=0x7fec1802a340 comp rx=0 tx=0).stop 2024-04-24T22:53:27.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 -- 172.21.15.111:0/1106938683 shutdown_connections 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 --2- 172.21.15.111:0/1106938683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28108600 0x7fec28114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 --2- 172.21.15.111:0/1106938683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28107c80 0x7fec281080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 --2- 172.21.15.111:0/1106938683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec2810dff0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.635+0000 7fec27200640 1 -- 172.21.15.111:0/1106938683 >> 172.21.15.111:0/1106938683 conn(0x7fec28077520 msgr2=0x7fec28106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.636+0000 7fec27200640 1 -- 172.21.15.111:0/1106938683 shutdown_connections 2024-04-24T22:53:27.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.636+0000 7fec27200640 1 -- 172.21.15.111:0/1106938683 wait complete. 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.636+0000 7fec27200640 1 Processor -- start 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 -- start start 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28107c80 0x7fec2819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec281a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fec28117f80 con 0x7fec28107c80 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fec28117e00 con 0x7fec2810dc10 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec27200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec28118100 con 0x7fec28108600 2024-04-24T22:53:27.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec25e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28107c80 0x7fec2819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec1d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec26800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec281a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec1d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53128/0 (socket says 172.21.15.111:53128) 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.637+0000 7fec1d400640 1 -- 172.21.15.111:0/571521244 learned_addr learned my addr 172.21.15.111:0/571521244 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 -- 172.21.15.111:0/571521244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 msgr2=0x7fec281a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec281a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 -- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28107c80 msgr2=0x7fec2819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28107c80 0x7fec2819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 -- 172.21.15.111:0/571521244 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec281a3410 con 0x7fec28108600 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1d400640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ea60 tx=0x7fec1000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1000cde0 con 0x7fec28108600 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec281a3700 con 0x7fec28108600 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec100035f0 con 0x7fec28108600 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10010670 con 0x7fec28108600 2024-04-24T22:53:27.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.638+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec281aaf90 con 0x7fec28108600 2024-04-24T22:53:27.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.639+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2804fd50 con 0x7fec28108600 2024-04-24T22:53:27.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.641+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fec100037a0 con 0x7fec28108600 2024-04-24T22:53:27.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.643+0000 7fec1f400640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 0x7fec0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:27.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.643+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fec100cf760 con 0x7fec28108600 2024-04-24T22:53:27.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.643+0000 7fec25e00640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 0x7fec0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:27.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.644+0000 7fec25e00640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 0x7fec0c0a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a930 tx=0x7fec180056d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:27.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.647+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec1009c320 con 0x7fec28108600 2024-04-24T22:53:27.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:27 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:27.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:27 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:27.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:27 smithi171 ceph-mon[32337]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:27.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:27 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:27.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:27 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:27.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:27 smithi186 ceph-mon[32753]: from='mgr.25204 ' entity='mgr.x' 2024-04-24T22:53:27.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.854+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fec28061b80 con 0x7fec0c0a2cf0 2024-04-24T22:53:27.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.870+0000 7fec1f400640 1 -- 172.21.15.111:0/571521244 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fec28061b80 con 0x7fec0c0a2cf0 2024-04-24T22:53:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T22:41:58", "first": "2024-04-24T22:42:00", "last": "2024-04-24T22:53:00", "last_pruned": "2024-04-24T22:53:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-04-24T22:53:27.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 msgr2=0x7fec0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:27.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 0x7fec0c0a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a930 tx=0x7fec180056d0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 msgr2=0x7fec2819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ea60 tx=0x7fec1000ef20 comp rx=0 tx=0).stop 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 shutdown_connections 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec2810dc10 0x7fec281a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fec0c0a2cf0 0x7fec0c0a51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec28108600 0x7fec2819e960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 --2- 172.21.15.111:0/571521244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec28107c80 0x7fec2819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:27.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 >> 172.21.15.111:0/571521244 conn(0x7fec28077520 msgr2=0x7fec28078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:27.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 shutdown_connections 2024-04-24T22:53:27.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:27.873+0000 7fec27200640 1 -- 172.21.15.111:0/571521244 wait complete. 2024-04-24T22:53:27.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 -- 172.21.15.111:0/622514465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 msgr2=0x7f3c78109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/622514465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c78109d30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009930 tx=0x7f3c6c02a420 comp rx=0 tx=0).stop 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 -- 172.21.15.111:0/622514465 shutdown_connections 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/622514465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c78110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/622514465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c78103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/622514465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c78109d30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 -- 172.21.15.111:0/622514465 >> 172.21.15.111:0/622514465 conn(0x7f3c780ff470 msgr2=0x7f3c78101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:28.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 -- 172.21.15.111:0/622514465 shutdown_connections 2024-04-24T22:53:28.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.074+0000 7f3c7fe00640 1 -- 172.21.15.111:0/622514465 wait complete. 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.075+0000 7f3c7fe00640 1 Processor -- start 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.075+0000 7f3c7fe00640 1 -- start start 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c7819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c7819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78113d60 con 0x7f3c78109950 2024-04-24T22:53:28.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78113be0 con 0x7f3c78103940 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78113ee0 con 0x7f3c78104300 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51432/0 (socket says 172.21.15.111:51432) 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7de00640 1 -- 172.21.15.111:0/2976805530 learned_addr learned my addr 172.21.15.111:0/2976805530 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7d400640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c7819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.076+0000 7f3c7ca00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c7819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 -- 172.21.15.111:0/2976805530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 msgr2=0x7f3c7819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c7819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 -- 172.21.15.111:0/2976805530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 msgr2=0x7f3c7819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c7819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 -- 172.21.15.111:0/2976805530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7819f240 con 0x7f3c78109950 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7ca00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c7819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7d400640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c7819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:53:28.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7de00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000eb10 tx=0x7f3c7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70004b20 con 0x7f3c78109950 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70004cc0 con 0x7f3c78109950 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7819f530 con 0x7f3c78109950 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.077+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70010760 con 0x7f3c78109950 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.078+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a6dc0 con 0x7f3c78109950 2024-04-24T22:53:28.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.079+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7804f500 con 0x7f3c78109950 2024-04-24T22:53:28.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.081+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3c70005020 con 0x7f3c78109950 2024-04-24T22:53:28.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.082+0000 7f3c76000640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 0x7f3c600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.082+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f3c700cfa80 con 0x7f3c78109950 2024-04-24T22:53:28.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.082+0000 7f3c7d400640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 0x7f3c600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.083+0000 7f3c7d400640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 0x7f3c600a52e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009900 tx=0x7f3c6c0058d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:28.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.087+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3c700d1e20 con 0x7f3c78109950 2024-04-24T22:53:28.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.311+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 --> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3c78061b80 con 0x7f3c600a2e20 2024-04-24T22:53:28.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.325+0000 7f3c76000640 1 -- 172.21.15.111:0/2976805530 <== mgr.25204 v2:172.21.15.111:6800/1932258627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3c78061b80 con 0x7f3c600a2e20 2024-04-24T22:53:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/ 1m 6m3h 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 msgr2=0x7f3c600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 0x7f3c600a52e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009900 tx=0x7f3c6c0058d0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 msgr2=0x7f3c7819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000eb10 tx=0x7f3c7000c2d0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 shutdown_connections 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78109950 0x7f3c7819eb10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f3c600a2e20 0x7f3c600a52e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c78104300 0x7f3c7819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 --2- 172.21.15.111:0/2976805530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78103940 0x7f3c7819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 >> 172.21.15.111:0/2976805530 conn(0x7f3c780ff470 msgr2=0x7f3c780ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:28.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 shutdown_connections 2024-04-24T22:53:28.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.328+0000 7f3c7fe00640 1 -- 172.21.15.111:0/2976805530 wait complete. 2024-04-24T22:53:28.348 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T22:53:28.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 -- 172.21.15.111:0/3452015402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 msgr2=0x7f0b0c112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 --2- 172.21.15.111:0/3452015402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c112080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009930 tx=0x7f0af802a440 comp rx=0 tx=0).stop 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 -- 172.21.15.111:0/3452015402 shutdown_connections 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 --2- 172.21.15.111:0/3452015402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c1125c0 0x7f0b0c1149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 --2- 172.21.15.111:0/3452015402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c112080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 --2- 172.21.15.111:0/3452015402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c075c70 0x7f0b0c076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 -- 172.21.15.111:0/3452015402 >> 172.21.15.111:0/3452015402 conn(0x7f0b0c0ff3f0 msgr2=0x7f0b0c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:28.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 -- 172.21.15.111:0/3452015402 shutdown_connections 2024-04-24T22:53:28.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.563+0000 7f0b12e00640 1 -- 172.21.15.111:0/3452015402 wait complete. 2024-04-24T22:53:28.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.564+0000 7f0b12e00640 1 Processor -- start 2024-04-24T22:53:28.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.564+0000 7f0b12e00640 1 -- start start 2024-04-24T22:53:28.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c1125c0 0x7f0b0c1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c118110 con 0x7f0b0c075c70 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c117f90 con 0x7f0b0c1125c0 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b12e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c118290 con 0x7f0b0c076590 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b0b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51452/0 (socket says 172.21.15.111:51452) 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b0be00640 1 -- 172.21.15.111:0/4210661377 learned_addr learned my addr 172.21.15.111:0/4210661377 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.565+0000 7f0b10e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c1125c0 0x7f0b0c1a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 -- 172.21.15.111:0/4210661377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 msgr2=0x7f0b0c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 -- 172.21.15.111:0/4210661377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c1125c0 msgr2=0x7f0b0c1a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c1125c0 0x7f0b0c1a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 -- 172.21.15.111:0/4210661377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1a3680 con 0x7f0b0c075c70 2024-04-24T22:53:28.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b0be00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00ea60 tx=0x7f0afc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc00cd20 con 0x7f0b0c075c70 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.566+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc0035f0 con 0x7f0b0c075c70 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.567+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a3970 con 0x7f0b0c075c70 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.567+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc010690 con 0x7f0b0c075c70 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.567+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1ab200 con 0x7f0b0c075c70 2024-04-24T22:53:28.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.568+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c068560 con 0x7f0b0c075c70 2024-04-24T22:53:28.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.570+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0afc0037a0 con 0x7f0b0c075c70 2024-04-24T22:53:28.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.571+0000 7f0b08c00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 0x7f0af00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:28.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.571+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f0afc0cf7b0 con 0x7f0b0c075c70 2024-04-24T22:53:28.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.571+0000 7f0b0b400640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 0x7f0af00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:28.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.572+0000 7f0b0b400640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 0x7f0af00a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009900 tx=0x7f0af80058d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:28.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.575+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0afc09c370 con 0x7f0b0c075c70 2024-04-24T22:53:28.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:28 smithi171 ceph-mon[32337]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:28.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:28 smithi186 ceph-mon[32753]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.777+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b0c10d7b0 con 0x7f0b0c075c70 2024-04-24T22:53:28.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.780+0000 7f0b08c00640 1 -- 172.21.15.111:0/4210661377 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23162 (secure 0 0 0) 0x7f0afc086090 con 0x7f0b0c075c70 2024-04-24T22:53:28.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:53:28.782 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":100,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:53:12.235227+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/1711063394":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2723701565":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/564429901":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2298793350":"2024-04-25T22:41:56.483825+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:53:28.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:28 smithi111 ceph-mon[27746]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-24T22:53:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 msgr2=0x7f0af00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 0x7f0af00a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009900 tx=0x7f0af80058d0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 msgr2=0x7f0b0c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00ea60 tx=0x7f0afc00ee60 comp rx=0 tx=0).stop 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 shutdown_connections 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c1125c0 0x7f0b0c1a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f0af00a2d00 0x7f0af00a51c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b0c076590 0x7f0b0c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 --2- 172.21.15.111:0/4210661377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b0c075c70 0x7f0b0c19e690 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 >> 172.21.15.111:0/4210661377 conn(0x7f0b0c0ff3f0 msgr2=0x7f0b0c113cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:28.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 shutdown_connections 2024-04-24T22:53:28.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:28.783+0000 7f0b12e00640 1 -- 172.21.15.111:0/4210661377 wait complete. 2024-04-24T22:53:28.804 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:53:28.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:53:29.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 -- 172.21.15.111:0/3268814710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010a170 msgr2=0x7fdd2010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 --2- 172.21.15.111:0/3268814710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010a170 0x7fdd2010a550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009860 tx=0x7fdd1002a340 comp rx=0 tx=0).stop 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 -- 172.21.15.111:0/3268814710 shutdown_connections 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 --2- 172.21.15.111:0/3268814710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010f240 0x7fdd20116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 --2- 172.21.15.111:0/3268814710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd2010ab20 0x7fdd2010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 --2- 172.21.15.111:0/3268814710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010a170 0x7fdd2010a550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 -- 172.21.15.111:0/3268814710 >> 172.21.15.111:0/3268814710 conn(0x7fdd20072d40 msgr2=0x7fdd20073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 -- 172.21.15.111:0/3268814710 shutdown_connections 2024-04-24T22:53:29.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.006+0000 7fdd27600640 1 -- 172.21.15.111:0/3268814710 wait complete. 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.007+0000 7fdd27600640 1 Processor -- start 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.007+0000 7fdd27600640 1 -- start start 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 0x7fdd201a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 0x7fdd201a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2011a2d0 con 0x7fdd2010a170 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2011a150 con 0x7fdd201a28e0 2024-04-24T22:53:29.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd27600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2011a450 con 0x7fdd2010f240 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51480/0 (socket says 172.21.15.111:51480) 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 -- 172.21.15.111:0/4034258791 learned_addr learned my addr 172.21.15.111:0/4034258791 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd17e00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 0x7fdd201a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd25600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 0x7fdd201a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 -- 172.21.15.111:0/4034258791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 msgr2=0x7fdd201a8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 0x7fdd201a8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 -- 172.21.15.111:0/4034258791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 msgr2=0x7fdd201a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 0x7fdd201a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd24c00640 1 -- 172.21.15.111:0/4034258791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201a3400 con 0x7fdd2010a170 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd17e00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 0x7fdd201a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.008+0000 7fdd25600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 0x7fdd201a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd24c00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdd10035040 tx=0x7fdd10033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10037040 con 0x7fdd2010a170 2024-04-24T22:53:29.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10040070 con 0x7fdd2010a170 2024-04-24T22:53:29.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201a3690 con 0x7fdd2010a170 2024-04-24T22:53:29.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100039b0 con 0x7fdd2010a170 2024-04-24T22:53:29.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.009+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201af8e0 con 0x7fdd2010a170 2024-04-24T22:53:29.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.011+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd20068560 con 0x7fdd2010a170 2024-04-24T22:53:29.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.012+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdd100305e0 con 0x7fdd2010a170 2024-04-24T22:53:29.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.013+0000 7fdd1e000640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 0x7fdd040a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.014+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fdd100ef3a0 con 0x7fdd2010a170 2024-04-24T22:53:29.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.014+0000 7fdd17e00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 0x7fdd040a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.015+0000 7fdd17e00640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 0x7fdd040a5240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd20117240 tx=0x7fdd08005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.018+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd100bc0d0 con 0x7fdd2010a170 2024-04-24T22:53:29.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.219+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd2010ab50 con 0x7fdd2010a170 2024-04-24T22:53:29.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.220+0000 7fdd1e000640 1 -- 172.21.15.111:0/4034258791 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fdd100bb9d0 con 0x7fdd2010a170 2024-04-24T22:53:29.221 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:53:29.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.222+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 msgr2=0x7fdd040a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.222+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 0x7fdd040a5240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd20117240 tx=0x7fdd08005f90 comp rx=0 tx=0).stop 2024-04-24T22:53:29.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 msgr2=0x7fdd201a8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdd10035040 tx=0x7fdd10033b60 comp rx=0 tx=0).stop 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 shutdown_connections 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd201a28e0 0x7fdd201a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fdd040a2d80 0x7fdd040a5240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd2010f240 0x7fdd201a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 --2- 172.21.15.111:0/4034258791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd2010a170 0x7fdd201a8730 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 >> 172.21.15.111:0/4034258791 conn(0x7fdd20072d40 msgr2=0x7fdd2006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 shutdown_connections 2024-04-24T22:53:29.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.223+0000 7fdd27600640 1 -- 172.21.15.111:0/4034258791 wait complete. 2024-04-24T22:53:29.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:53:29.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 -- 172.21.15.111:0/488681550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 msgr2=0x7f5684109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 --2- 172.21.15.111:0/488681550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f5684109be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5674009860 tx=0x7f567402a340 comp rx=0 tx=0).stop 2024-04-24T22:53:29.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 -- 172.21.15.111:0/488681550 shutdown_connections 2024-04-24T22:53:29.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 --2- 172.21.15.111:0/488681550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841041b0 0x7f56841106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 --2- 172.21.15.111:0/488681550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841037f0 0x7f5684103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 --2- 172.21.15.111:0/488681550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f5684109be0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 -- 172.21.15.111:0/488681550 >> 172.21.15.111:0/488681550 conn(0x7f56840ff280 msgr2=0x7f56841016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:29.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 -- 172.21.15.111:0/488681550 shutdown_connections 2024-04-24T22:53:29.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.451+0000 7f5683e00640 1 -- 172.21.15.111:0/488681550 wait complete. 2024-04-24T22:53:29.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.453+0000 7f5683e00640 1 Processor -- start 2024-04-24T22:53:29.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.453+0000 7f5683e00640 1 -- start start 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.453+0000 7f5683e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.453+0000 7f5683e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841041b0 0x7f568419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.453+0000 7f5683e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f568419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5683e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5684113c10 con 0x7f56841041b0 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5683e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5684113a90 con 0x7f5684109800 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5683e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5684113d90 con 0x7f56841037f0 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:53194/0 (socket says 172.21.15.111:53194) 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841041b0 0x7f568419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 -- 172.21.15.111:0/3183569390 learned_addr learned my addr 172.21.15.111:0/3183569390 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5683400640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f568419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 -- 172.21.15.111:0/3183569390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 msgr2=0x7f568419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f568419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 -- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841041b0 msgr2=0x7f568419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841041b0 0x7f568419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.454+0000 7f5682a00640 1 -- 172.21.15.111:0/3183569390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568419f0c0 con 0x7f56841037f0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5682a00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5674035040 tx=0x7f5674033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567403f020 con 0x7f56841037f0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5674004040 con 0x7f56841037f0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568419f350 con 0x7f56841037f0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674037040 con 0x7f56841037f0 2024-04-24T22:53:29.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.455+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841a6c40 con 0x7f56841037f0 2024-04-24T22:53:29.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.456+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5684068480 con 0x7f56841037f0 2024-04-24T22:53:29.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.458+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5674042020 con 0x7f56841037f0 2024-04-24T22:53:29.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.459+0000 7f5688a00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 0x7f56680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.460+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f56740ef450 con 0x7f56841037f0 2024-04-24T22:53:29.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.460+0000 7f5682000640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 0x7f56680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.460+0000 7f5682000640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 0x7f56680a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5684108e00 tx=0x7f5670009260 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.462+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5674049020 con 0x7f56841037f0 2024-04-24T22:53:29.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:29 smithi111 ceph-mon[27746]: from='client.26188 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:29 smithi111 ceph-mon[27746]: from='client.16146 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:29 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4210661377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:53:29.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:29 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/4034258791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:53:29.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:29 smithi171 ceph-mon[32337]: from='client.26188 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:29 smithi171 ceph-mon[32337]: from='client.16146 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:29 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/4210661377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:53:29.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:29 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/4034258791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:53:29.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.660+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f568410bd60 con 0x7f56841037f0 2024-04-24T22:53:29.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.661+0000 7f5688a00640 1 -- 172.21.15.111:0/3183569390 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f56740bc120 con 0x7f56841037f0 2024-04-24T22:53:29.663 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 msgr2=0x7f56680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 0x7f56680a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5684108e00 tx=0x7f5670009260 comp rx=0 tx=0).stop 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 msgr2=0x7f568419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5674035040 tx=0x7f5674033b60 comp rx=0 tx=0).stop 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 shutdown_connections 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5684109800 0x7f568419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56841041b0 0x7f568419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f56680a2d00 0x7f56680a51c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 --2- 172.21.15.111:0/3183569390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56841037f0 0x7f568419a0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 >> 172.21.15.111:0/3183569390 conn(0x7f56840ff280 msgr2=0x7f56840ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:29.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 shutdown_connections 2024-04-24T22:53:29.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.664+0000 7f5683e00640 1 -- 172.21.15.111:0/3183569390 wait complete. 2024-04-24T22:53:29.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:53:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:29 smithi186 ceph-mon[32753]: from='client.26188 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:29 smithi186 ceph-mon[32753]: from='client.16146 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:29 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/4210661377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:53:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:29 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/4034258791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 -- 172.21.15.111:0/479251368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8073b70 msgr2=0x7f08d8073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 --2- 172.21.15.111:0/479251368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8073b70 0x7f08d8073f50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009860 tx=0x7f08cc02a3d0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 -- 172.21.15.111:0/479251368 shutdown_connections 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 --2- 172.21.15.111:0/479251368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 0x7f08d81103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 --2- 172.21.15.111:0/479251368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8074490 0x7f08d810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 --2- 172.21.15.111:0/479251368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8073b70 0x7f08d8073f50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 -- 172.21.15.111:0/479251368 >> 172.21.15.111:0/479251368 conn(0x7f08d80fd250 msgr2=0x7f08d80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 -- 172.21.15.111:0/479251368 shutdown_connections 2024-04-24T22:53:29.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.890+0000 7f08e0600640 1 -- 172.21.15.111:0/479251368 wait complete. 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.891+0000 7f08e0600640 1 Processor -- start 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.891+0000 7f08e0600640 1 -- start start 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8074490 0x7f08d819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 0x7f08d819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8113d10 con 0x7f08d810dfa0 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8113b90 con 0x7f08d8073b70 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08e0600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8113e90 con 0x7f08d8074490 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08ddc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08de600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 0x7f08d819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08ddc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:52804/0 (socket says 172.21.15.111:52804) 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08ddc00640 1 -- 172.21.15.111:0/1233611496 learned_addr learned my addr 172.21.15.111:0/1233611496 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.892+0000 7f08dd200640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8074490 0x7f08d819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 -- 172.21.15.111:0/1233611496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8074490 msgr2=0x7f08d819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8074490 0x7f08d819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 -- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 msgr2=0x7f08d819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 0x7f08d819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 -- 172.21.15.111:0/1233611496 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d819f2c0 con 0x7f08d8073b70 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08ddc00640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08cc02a8f0 tx=0x7f08cc030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc040020 con 0x7f08d8073b70 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d819f550 con 0x7f08d8073b70 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08cc004460 con 0x7f08d8073b70 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.893+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0046c0 con 0x7f08d8073b70 2024-04-24T22:53:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.894+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d81a6e40 con 0x7f08d8073b70 2024-04-24T22:53:29.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.895+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d8075220 con 0x7f08d8073b70 2024-04-24T22:53:29.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.897+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f08cc043020 con 0x7f08d8073b70 2024-04-24T22:53:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.898+0000 7f08d6000640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 0x7f08c00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.898+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f08cc0f04b0 con 0x7f08d8073b70 2024-04-24T22:53:29.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.898+0000 7f08dd200640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 0x7f08c00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:29.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.899+0000 7f08dd200640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 0x7f08c00a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ff950 tx=0x7f08c4005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:29.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:29.901+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08cc04a020 con 0x7f08d8073b70 2024-04-24T22:53:30.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.105+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08d8109120 con 0x7f08d8073b70 2024-04-24T22:53:30.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.106+0000 7f08d6000640 1 -- 172.21.15.111:0/1233611496 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f08cc0bd070 con 0x7f08d8073b70 2024-04-24T22:53:30.107 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:53:30.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.108+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 msgr2=0x7f08c00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:30.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.108+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 0x7f08c00a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ff950 tx=0x7f08c4005f90 comp rx=0 tx=0).stop 2024-04-24T22:53:30.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.108+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 msgr2=0x7f08d819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:30.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.108+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08cc02a8f0 tx=0x7f08cc030650 comp rx=0 tx=0).stop 2024-04-24T22:53:30.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 shutdown_connections 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d810dfa0 0x7f08d819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f08c00a2d00 0x7f08c00a51c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08d8074490 0x7f08d819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 --2- 172.21.15.111:0/1233611496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08d8073b70 0x7f08d819a2d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 >> 172.21.15.111:0/1233611496 conn(0x7f08d80fd250 msgr2=0x7f08d80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 shutdown_connections 2024-04-24T22:53:30.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.109+0000 7f08e0600640 1 -- 172.21.15.111:0/1233611496 wait complete. 2024-04-24T22:53:30.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 -- 172.21.15.111:0/1054461678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 msgr2=0x7f2768103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 --2- 172.21.15.111:0/1054461678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f2768103da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2750009930 tx=0x7f275002a420 comp rx=0 tx=0).stop 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 -- 172.21.15.111:0/1054461678 shutdown_connections 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 --2- 172.21.15.111:0/1054461678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27681042e0 0x7f27681107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 --2- 172.21.15.111:0/1054461678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f2768103da0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 --2- 172.21.15.111:0/1054461678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2768109930 0x7f2768109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.328+0000 7f276e000640 1 -- 172.21.15.111:0/1054461678 >> 172.21.15.111:0/1054461678 conn(0x7f27680ff3f0 msgr2=0x7f2768101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:30.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.329+0000 7f276e000640 1 -- 172.21.15.111:0/1054461678 shutdown_connections 2024-04-24T22:53:30.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.329+0000 7f276e000640 1 -- 172.21.15.111:0/1054461678 wait complete. 2024-04-24T22:53:30.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.329+0000 7f276e000640 1 Processor -- start 2024-04-24T22:53:30.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 -- start start 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 0x7f276819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2768109930 0x7f276819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113d90 con 0x7f2768103920 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113c10 con 0x7f27681042e0 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f276e000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113f10 con 0x7f2768109930 2024-04-24T22:53:30.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f2767400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f2767400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51534/0 (socket says 172.21.15.111:51534) 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f2767400640 1 -- 172.21.15.111:0/3609253047 learned_addr learned my addr 172.21.15.111:0/3609253047 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.330+0000 7f2766a00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 0x7f276819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767e00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2768109930 0x7f276819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 -- 172.21.15.111:0/3609253047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2768109930 msgr2=0x7f276819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2768109930 0x7f276819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 -- 172.21.15.111:0/3609253047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 msgr2=0x7f276819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 0x7f276819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 -- 172.21.15.111:0/3609253047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276819f240 con 0x7f2768103920 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2766a00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 0x7f276819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f2767400640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f275800e9f0 tx=0x7f275800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275800cd80 con 0x7f2768103920 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276819f530 con 0x7f2768103920 2024-04-24T22:53:30.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27681a6dc0 con 0x7f2768103920 2024-04-24T22:53:30.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2758010910 con 0x7f2768103920 2024-04-24T22:53:30.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.331+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758018980 con 0x7f2768103920 2024-04-24T22:53:30.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.332+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744005290 con 0x7f2768103920 2024-04-24T22:53:30.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.335+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2758010ac0 con 0x7f2768103920 2024-04-24T22:53:30.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.337+0000 7f275fe00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 0x7f274c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:30.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.337+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f27580cfa40 con 0x7f2768103920 2024-04-24T22:53:30.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.337+0000 7f2766a00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 0x7f274c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:30.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.338+0000 7f2766a00640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 0x7f274c0a5240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2750006010 tx=0x7f2750005a20 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:30.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.340+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f275809c600 con 0x7f2768103920 2024-04-24T22:53:30.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:30 smithi111 ceph-mon[27746]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-24T22:53:30.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:30 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3183569390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:53:30.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:30 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1233611496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:53:30.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.580+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2744005850 con 0x7f2768103920 2024-04-24T22:53:30.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.613+0000 7f275fe00640 1 -- 172.21.15.111:0/3609253047 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740910 (secure 0 0 0) 0x7f275809bf00 con 0x7f2768103920 2024-04-24T22:53:30.625 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:53:30.625 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:53:30.625 INFO:teuthology.orchestra.run.smithi111.stdout: "epoch": 26, 2024-04-24T22:53:30.625 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "active_gid": 25204, 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "active_name": "x", 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "active_addrs": { 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:6800", 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1932258627 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:6801", 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1932258627 2024-04-24T22:53:30.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "active_addr": "172.21.15.111:6801/1932258627", 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "active_change": "2024-04-24T22:41:56.484131+0000", 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "available": true, 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "standbys": [ 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "gid": 25184, 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "y", 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr_features": 4540701547738038271, 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.627 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.628 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.629 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.630 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.631 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.632 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.633 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-04-24T22:53:30.636 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.637 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.638 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.639 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.640 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.641 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.642 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.644 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-04-24T22:53:30.645 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-04-24T22:53:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.647 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-04-24T22:53:30.648 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.650 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.651 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.652 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.653 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.655 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-04-24T22:53:30.656 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T22:53:30.657 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.658 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.659 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.660 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.661 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.662 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T22:53:30.664 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.665 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T22:53:30.667 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.668 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-04-24T22:53:30.670 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-04-24T22:53:30.671 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T22:53:30.673 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.674 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T22:53:30.675 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.677 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.678 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.679 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-04-24T22:53:30.681 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-04-24T22:53:30.682 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.683 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.684 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.685 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:30.686 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.687 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.688 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-04-24T22:53:30.689 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.690 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.691 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.692 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-04-24T22:53:30.693 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.694 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.695 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.696 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.697 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.698 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.699 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.700 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.701 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.702 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.703 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.704 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.705 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.706 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T22:53:30.707 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.708 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.709 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.710 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-04-24T22:53:30.712 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-04-24T22:53:30.713 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.714 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.715 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.716 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T22:53:30.718 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.719 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.722 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.724 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.727 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.728 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.729 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.731 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.732 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T22:53:30.733 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T22:53:30.734 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.735 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.736 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.739 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.740 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.743 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.745 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T22:53:30.746 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T22:53:30.747 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.748 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.749 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.750 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.751 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.752 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.753 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.754 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.755 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.757 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.758 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T22:53:30.759 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T22:53:30.760 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.761 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.762 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.763 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.764 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.765 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.766 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.768 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.769 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:30.770 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.771 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.772 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.773 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.776 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.777 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.778 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.780 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:30.781 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.783 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.784 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.785 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.786 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.787 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.789 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-04-24T22:53:30.790 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.791 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-04-24T22:53:30.792 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.793 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.794 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.795 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.796 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.798 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.800 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.802 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.803 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.805 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.806 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-04-24T22:53:30.807 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.808 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.809 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.810 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.811 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.812 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.813 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-04-24T22:53:30.814 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-04-24T22:53:30.816 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-04-24T22:53:30.817 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.818 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.819 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.820 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.822 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.823 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.824 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.825 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.826 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.827 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.828 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-04-24T22:53:30.829 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.830 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.831 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.832 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.833 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.834 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.835 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-04-24T22:53:30.836 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.837 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.838 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.839 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.840 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.841 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.842 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.843 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T22:53:30.844 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.845 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T22:53:30.846 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.847 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.849 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.850 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.851 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.852 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.854 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.855 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.856 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.857 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.861 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.862 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.863 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.864 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.865 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.866 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.867 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.868 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.869 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.870 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.871 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.872 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.874 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.875 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.876 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.877 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.878 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.879 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.880 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.881 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.883 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:30.884 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-04-24T22:53:30.885 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.886 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:30.887 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.888 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.889 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.891 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.892 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-04-24T22:53:30.893 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-04-24T22:53:30.894 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.895 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.896 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.897 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.898 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.900 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.901 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.902 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-04-24T22:53:30.903 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.904 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.905 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.907 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.908 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.909 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.910 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.911 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.912 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.913 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.914 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.915 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.916 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.917 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.918 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.919 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.920 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.921 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.922 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.923 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.924 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.925 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.926 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.927 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.928 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.929 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.930 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.931 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.933 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.934 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.935 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T22:53:30.936 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.937 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.938 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.939 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.940 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.941 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.942 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.943 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.944 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-04-24T22:53:30.946 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.947 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.948 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.951 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.953 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.954 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.955 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.956 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.958 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.959 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.960 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.962 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.963 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.965 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.966 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.968 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.969 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.970 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-04-24T22:53:30.971 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.972 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.973 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.974 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.975 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.976 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.978 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.979 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.980 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-04-24T22:53:30.982 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-04-24T22:53:30.983 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:30.984 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.985 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.986 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.987 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.988 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.989 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.990 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.991 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-04-24T22:53:30.993 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.995 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:30.996 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.998 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.000 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.001 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.002 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.003 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.005 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.006 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.007 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.008 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.009 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-04-24T22:53:31.010 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.011 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.012 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.013 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.015 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.016 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.017 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.020 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "gid": 25201, 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "z", 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr_features": 4540701547738038271, 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.021 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.022 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.023 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.024 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.025 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.026 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.027 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.028 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-04-24T22:53:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.032 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.033 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.034 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.035 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T22:53:31.036 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.040 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-04-24T22:53:31.041 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.042 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.043 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.044 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.045 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.046 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T22:53:31.047 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-04-24T22:53:31.048 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.049 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.051 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T22:53:31.052 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.053 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-04-24T22:53:31.054 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.055 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.055 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.055 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.057 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.060 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.061 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.062 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.064 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.066 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-04-24T22:53:31.067 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.070 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.072 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-04-24T22:53:31.073 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-04-24T22:53:31.074 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.075 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.076 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T22:53:31.078 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.079 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T22:53:31.080 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T22:53:31.081 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T22:53:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.083 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.085 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-04-24T22:53:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-04-24T22:53:31.087 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.088 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.089 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.091 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.092 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.093 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.094 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.095 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.096 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.097 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.098 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.099 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:31.100 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.101 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-04-24T22:53:31.102 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-04-24T22:53:31.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.104 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.105 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.106 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.107 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.108 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.110 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T22:53:31.111 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.112 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.113 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.114 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.115 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.116 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.117 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.118 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.120 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.121 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.122 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.123 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.124 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.125 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.126 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.127 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.128 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.130 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.131 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.134 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.135 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.136 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-04-24T22:53:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.138 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.139 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.140 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.141 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.142 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.143 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.144 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.145 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.146 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.147 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.148 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.150 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T22:53:31.151 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.155 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.156 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.157 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.161 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.162 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-04-24T22:53:31.163 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.164 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.165 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.166 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.167 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.168 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.169 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.170 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.171 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-04-24T22:53:31.172 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.173 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.174 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.175 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-04-24T22:53:31.176 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.177 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.178 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-04-24T22:53:31.179 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.182 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.183 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.184 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-04-24T22:53:31.186 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.187 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.188 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.189 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.190 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.191 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-04-24T22:53:31.192 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.193 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.194 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.195 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.196 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.197 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.198 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-04-24T22:53:31.199 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.200 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-04-24T22:53:31.201 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-04-24T22:53:31.202 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-04-24T22:53:31.203 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.206 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.207 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.208 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.209 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.210 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-04-24T22:53:31.211 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.212 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.214 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.217 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.218 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.219 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.221 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.222 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.224 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.225 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.226 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.227 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.228 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.229 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.232 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.233 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.234 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.235 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.237 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.238 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T22:53:31.239 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.241 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.242 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T22:53:31.243 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.244 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.245 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.246 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.249 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.250 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.252 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.253 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.254 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.255 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.256 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.257 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.258 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-04-24T22:53:31.259 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-04-24T22:53:31.260 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.261 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.262 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.263 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.264 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.265 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.266 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.267 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.268 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.269 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.270 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.271 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.272 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.273 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.275 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.276 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.277 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.278 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.279 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.280 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.281 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.282 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.283 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.284 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.285 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.286 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.287 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.288 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.289 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.290 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.291 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.292 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.293 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.294 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.295 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.296 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.298 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.299 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.300 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.301 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.302 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.305 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.306 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.307 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.308 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.309 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.310 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.311 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.312 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.313 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.314 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.315 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.316 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.317 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.319 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-04-24T22:53:31.320 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.321 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.322 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.323 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.325 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.326 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.327 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.328 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.330 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.332 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.333 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.334 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.337 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.338 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.339 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.340 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.341 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.343 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.344 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.346 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.347 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-04-24T22:53:31.348 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-04-24T22:53:31.349 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-04-24T22:53:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.353 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.354 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.355 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-04-24T22:53:31.356 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.357 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.358 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.360 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.361 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.362 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.363 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.364 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.366 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.367 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.368 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.369 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.370 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.371 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.372 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.374 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.376 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.378 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.379 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-04-24T22:53:31.381 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-04-24T22:53:31.382 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.383 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.384 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.386 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.387 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.388 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.389 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.390 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.391 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.393 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.394 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.395 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.396 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.397 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.398 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.399 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.400 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.401 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.402 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.403 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.404 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.405 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.406 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.407 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.408 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.409 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.410 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.411 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.412 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.413 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.414 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.415 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.416 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.417 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.418 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.419 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.420 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.421 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-04-24T22:53:31.423 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.424 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "modules": [ 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "dashboard", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "iostat", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "nfs", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "restful", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "snap_schedule" 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.425 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.427 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.428 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.429 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.430 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.431 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.432 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.433 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-04-24T22:53:31.434 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.435 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.436 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-04-24T22:53:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.438 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.439 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-04-24T22:53:31.443 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-04-24T22:53:31.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.445 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.446 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.448 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.449 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.450 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.451 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-04-24T22:53:31.452 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.453 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.454 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.455 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T22:53:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.457 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.459 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.460 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.461 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-04-24T22:53:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-04-24T22:53:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-04-24T22:53:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-04-24T22:53:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-04-24T22:53:31.470 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-04-24T22:53:31.471 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-04-24T22:53:31.472 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-04-24T22:53:31.473 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-04-24T22:53:31.474 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.475 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.476 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.477 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.478 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.479 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T22:53:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.481 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.482 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.485 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.486 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.487 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.488 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.489 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-04-24T22:53:31.490 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.491 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.493 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T22:53:31.494 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.495 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.496 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T22:53:31.497 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.498 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.499 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.501 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.502 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.503 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.504 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.505 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.507 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.508 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-04-24T22:53:31.511 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.512 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-04-24T22:53:31.513 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-04-24T22:53:31.514 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-04-24T22:53:31.515 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.516 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.517 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.518 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.519 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.521 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.522 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-04-24T22:53:31.523 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.524 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.525 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.526 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.527 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.528 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.529 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.531 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.533 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.534 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.535 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-04-24T22:53:31.536 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-04-24T22:53:31.537 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.539 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.540 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.541 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T22:53:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T22:53:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T22:53:31.546 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.547 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.549 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.550 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.551 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.552 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.553 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.554 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.556 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T22:53:31.557 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.559 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.560 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.561 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T22:53:31.562 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.563 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.564 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.566 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.567 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.568 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.569 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.570 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.571 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T22:53:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.574 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.575 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.576 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.577 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.580 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.581 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.582 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.583 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.585 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.586 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:31.587 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.588 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.589 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.590 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.591 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.593 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.594 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.595 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.596 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.597 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.598 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.599 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.600 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.602 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.603 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-04-24T22:53:31.604 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.605 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.606 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.607 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.608 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.609 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.610 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.613 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.615 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-04-24T22:53:31.616 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.617 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-04-24T22:53:31.618 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.620 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.621 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.622 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.624 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.625 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.626 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.627 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.629 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-04-24T22:53:31.630 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.631 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.632 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.633 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.634 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.635 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.636 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.637 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.638 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.639 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.640 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.641 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-04-24T22:53:31.642 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.644 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.646 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.647 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.648 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.649 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.650 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.652 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.653 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.654 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.655 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.656 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.659 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.660 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.662 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-04-24T22:53:31.663 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.664 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.665 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.666 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.667 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.668 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-04-24T22:53:31.670 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T22:53:31.673 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.673 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.674 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.675 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.676 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.677 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.678 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.679 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.680 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.682 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.683 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.684 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.685 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.686 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.687 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.688 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.689 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.690 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.691 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.692 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.693 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.694 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-04-24T22:53:31.695 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.696 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.697 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.698 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.699 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.700 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.701 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.702 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.703 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.704 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.705 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.706 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.706 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.706 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.706 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.707 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.708 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.709 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.710 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.711 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.714 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.716 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.717 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.718 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.718 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.718 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.718 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.719 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.720 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.721 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.722 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.723 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.727 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.728 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.730 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.731 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.732 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.733 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.734 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.737 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.738 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.739 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.740 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.741 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.742 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.743 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.744 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.746 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.747 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.749 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.750 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.751 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.752 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.753 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.754 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-04-24T22:53:31.757 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.758 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.759 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.760 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.763 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.765 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.766 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-04-24T22:53:31.767 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-04-24T22:53:31.768 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.769 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.771 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.772 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.773 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.774 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.775 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.778 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.779 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.780 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.782 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.783 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.784 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.785 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.786 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.788 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-04-24T22:53:31.789 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-04-24T22:53:31.790 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-04-24T22:53:31.791 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.792 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.794 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.795 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.796 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.797 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-04-24T22:53:31.798 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.799 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.800 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.801 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.802 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.803 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.804 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.805 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.806 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.807 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.809 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.810 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.811 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.812 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.816 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.818 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.819 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-04-24T22:53:31.823 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.824 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.827 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-04-24T22:53:31.828 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-04-24T22:53:31.829 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.830 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.831 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-04-24T22:53:31.832 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.833 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.834 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.835 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.836 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.838 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.839 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.841 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.842 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.843 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.844 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.845 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.846 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.847 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.848 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.849 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.850 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.851 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.852 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.853 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.854 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.855 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.857 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.858 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-04-24T22:53:31.859 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.860 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.861 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.862 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.863 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.864 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.865 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.866 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.867 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.868 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-04-24T22:53:31.869 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-04-24T22:53:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.871 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "services": { 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "dashboard": "https://172.21.15.111:8443/" 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "always_on_modules": { 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "octopus": [ 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-04-24T22:53:31.872 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "pacific": [ 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "quincy": [ 2024-04-24T22:53:31.873 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "reef": [ 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-04-24T22:53:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "squid": [ 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-04-24T22:53:31.875 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "last_failure_osd_epoch": 88, 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "active_clients": [ 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1374232862 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.876 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "libcephsqlite", 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1189193056 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1019118003 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 912252228 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1100451211 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-04-24T22:53:31.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 77638538 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-24T22:53:31.880 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:53:31.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:30 smithi186 ceph-mon[32753]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:30 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3183569390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:30 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1233611496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:31 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/3609253047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:30 smithi171 ceph-mon[32337]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:30 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3183569390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:30 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1233611496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:53:31.882 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:31 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/3609253047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.621+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 msgr2=0x7f274c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.621+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 0x7f274c0a5240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2750006010 tx=0x7f2750005a20 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.621+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 msgr2=0x7f276819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.621+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f275800e9f0 tx=0x7f275800eeb0 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 shutdown_connections 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f274c0a2d80 0x7f274c0a5240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2768109930 0x7f276819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27681042e0 0x7f276819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 --2- 172.21.15.111:0/3609253047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2768103920 0x7f276819a250 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 >> 172.21.15.111:0/3609253047 conn(0x7f27680ff3f0 msgr2=0x7f27680ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 shutdown_connections 2024-04-24T22:53:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:53:30.622+0000 7f276e000640 1 -- 172.21.15.111:0/3609253047 wait complete. 2024-04-24T22:53:31.890 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T22:53:31.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T22:53:31.890 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:31 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/3609253047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T22:53:32.336 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:53:32.539 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:32 smithi111 ceph-mon[27746]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-24T22:53:32.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:32 smithi171 ceph-mon[32337]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-24T22:53:32.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:32 smithi186 ceph-mon[32753]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-24T22:53:34.637 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:34 smithi111 ceph-mon[27746]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 0, 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 2, 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 0, 2024-04-24T22:53:34.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.004604343, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.001224504, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:34.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 6, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 176, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 3344, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 186480, 2024-04-24T22:53:34.643 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.038196944, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.028161519, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003953391, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 174, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.025529383, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000146720 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:34.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 186480 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 238, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 227, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1585280, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 140064, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.097530268, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.035722864, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.066431315, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.010808224, 2024-04-24T22:53:34.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 226, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.022363429, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000098953 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1585280, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 140064 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-04-24T22:53:34.646 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:34.647 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:34.648 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 151039, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 32, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:34.649 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-04-24T22:53:34.650 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 240, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1551745, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 240, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 240, 2024-04-24T22:53:34.651 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1551745, 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:34.652 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:53:34.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:34 smithi171 ceph-mon[32337]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-04-24T22:53:34.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:34 smithi186 ceph-mon[32753]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-04-24T22:53:35.482 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T22:53:35.482 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T22:53:35.482 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-24T22:53:35.482 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-24T22:53:35.482 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-24T22:53:35.482 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T22:53:35.483 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T22:53:35.895 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:53:36.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:36 smithi111 ceph-mon[27746]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:36.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:36 smithi171 ceph-mon[32337]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:36.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:36 smithi186 ceph-mon[32753]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:38.128 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:53:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 0, 2024-04-24T22:53:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 2, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 0, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.003778465, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.000617331, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 180, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 320, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 187088, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.036895509, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.027414233, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003157399, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 177, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026984113, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000152452 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 187088 2024-04-24T22:53:38.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 249, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 236, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1590208, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 144992, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.100673512, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.036266917, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.069943171, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.011127717, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 236, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.024416827, 2024-04-24T22:53:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000103461 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1590208, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 144992 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:38.133 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:38.134 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:38.135 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 148977, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 30, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:38.136 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 247, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1552655, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 247, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 247, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1552655, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:38.138 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:38.139 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:53:38.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:38 smithi111 ceph-mon[27746]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:38.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:38 smithi171 ceph-mon[32337]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:38.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:38 smithi186 ceph-mon[32753]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-24T22:53:39.006 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T22:53:39.006 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T22:53:39.006 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-24T22:53:39.006 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-24T22:53:39.006 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-24T22:53:39.006 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T22:53:39.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T22:53:39.459 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:53:40.641 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:40 smithi111 ceph-mon[27746]: pgmap v366: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-24T22:53:40.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:40 smithi171 ceph-mon[32337]: pgmap v366: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-24T22:53:40.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:40 smithi186 ceph-mon[32753]: pgmap v366: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-24T22:53:41.684 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:53:41.684 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:41.684 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 920, 2024-04-24T22:53:41.684 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 901, 2024-04-24T22:53:41.684 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1788290, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 387710, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 7, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.257365855, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.088308683, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.190883730, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.020038927, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 900, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.094205811, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000104673 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:41.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1207, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000534504, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000442 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1627136, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 150256 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 27, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 203, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 8552, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 531173, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 14, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-04-24T22:53:41.686 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.044512195, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.027975929, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.010005121, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000323937, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 201, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.043430317, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000216071 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 19, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000003435, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000180 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 5184, 2024-04-24T22:53:41.687 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 478624 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 26, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 46, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 6429, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 19760, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 7, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.013393078, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.005023336, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.007102981, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000671918, 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:41.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 44, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.039894451, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000906692 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 36, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000016915, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000469 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 6432 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-24T22:53:41.689 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 43, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002123670, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000049387 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-24T22:53:41.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000007189, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000003594 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 3, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 3, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002933018, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000977672 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-24T22:53:41.691 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 11, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 11, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 10, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 1, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-24T22:53:41.692 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 8, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 4, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 2, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-24T22:53:41.693 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 5, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 298, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-24T22:53:41.694 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 3, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-24T22:53:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.700 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 5, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 5, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-24T22:53:41.701 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195815, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.009749560, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002437390 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 14, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 14, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 13, 2024-04-24T22:53:41.702 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 14, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 11, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 12, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 1, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 2, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 2, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 50560, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223612 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 8, 2024-04-24T22:53:41.703 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 3, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 13, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 1, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001923133, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001923133 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001009885, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000504942 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-24T22:53:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-24T22:53:41.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 2, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 1, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 2, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 2, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 698, 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-24T22:53:41.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:41.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:41.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 255041, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 88, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8032, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 167, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 56344, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 227, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-24T22:53:41.719 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 42, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 11958, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 42, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 42, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.289994964, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.006904642 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 42, 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 66 2024-04-24T22:53:41.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 42, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 42, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 4, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 7, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-24T22:53:41.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:53:41.722 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-24T22:53:41.723 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 100, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 15, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 17, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 2 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-24T22:53:41.724 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-24T22:53:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 969, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1697392, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 969, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 969, 2024-04-24T22:53:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1697392, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:41.727 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 42, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 11958, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 42, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 11958, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-24T22:53:41.728 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 42, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 42, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 42, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 42, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-24T22:53:41.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x563013882500": { 2024-04-24T22:53:41.730 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 5, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1511, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 5, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 4, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1511, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.731 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:41.732 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:53:42.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:42 smithi111 ceph-mon[27746]: pgmap v367: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-24T22:53:42.640 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-24T22:53:42.640 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-24T22:53:42.640 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-24T22:53:42.640 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-24T22:53:42.640 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T22:53:42.640 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T22:53:42.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T22:53:42.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:42 smithi171 ceph-mon[32337]: pgmap v367: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-24T22:53:42.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:42 smithi186 ceph-mon[32753]: pgmap v367: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-24T22:53:43.091 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T22:53:44.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:44 smithi111 ceph-mon[27746]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-24T22:53:44.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:44 smithi171 ceph-mon[32337]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-24T22:53:44.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:44 smithi186 ceph-mon[32753]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-24T22:53:45.384 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-24T22:53:45.384 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:45.384 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 252, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 239, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1591024, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 146912, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.109148527, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.036854710, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.080326969, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.011024865, 2024-04-24T22:53:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 239, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.024581476, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000102851 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1591024, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 146912 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:45.386 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 181, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 320, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 187264, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.039096025, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.029611231, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003043377, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 178, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.037406210, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000210147 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.387 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 187264 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 0, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 2, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 0, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.001250923, 2024-04-24T22:53:45.388 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.000985831, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:45.389 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:45.390 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:45.391 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 149745, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 31, 2024-04-24T22:53:45.392 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-04-24T22:53:45.393 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 250, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1553045, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 250, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-24T22:53:45.394 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 250, 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1553045, 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-24T22:53:45.395 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-24T22:53:46.205 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T22:53:46.205 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T22:53:46.205 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-24T22:53:46.205 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-24T22:53:46.206 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-24T22:53:46.206 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T22:53:46.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T22:53:46.457 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:53:46.578 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:46 smithi171 ceph-mon[32337]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:46.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:46 smithi186 ceph-mon[32753]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:46.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:46 smithi111 ceph-mon[27746]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2170, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2298, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 2179439, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 5433507, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 21, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-04-24T22:53:48.065 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.246239644, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.082367201, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.094869148, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.057635243, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2300, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.137858978, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000059938 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1172, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000276728, 2024-04-24T22:53:48.066 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000236 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1634768, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 499008 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1487, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1350, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 654369, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 4706510, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 103, 2024-04-24T22:53:48.067 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.205635412, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.050999334, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.082533951, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.054466139, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1352, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.227638119, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000168371 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1491, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000304366, 2024-04-24T22:53:48.068 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000204 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 88272, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 143248 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 5609, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 6609, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1810923, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 19921418, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 16, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 9, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.609980509, 2024-04-24T22:53:48.069 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.218477659, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.228212268, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.162292729, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6600, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.261739897, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000039657 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4136, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.001606655, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000388 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.070 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 426752, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 138736 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6307, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.945281655, 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000308432 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 16, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000498148, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000031134 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 781, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 781, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 781, 2024-04-24T22:53:48.072 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 4.127960285, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.005285480 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 349, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 42, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 15, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 27, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-04-24T22:53:48.073 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 20, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 3, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 4, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 2, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 816, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 810, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-04-24T22:53:48.074 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-04-24T22:53:48.075 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 616, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 612, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 599, 2024-04-24T22:53:48.076 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 734, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 713, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 2, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 621 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-04-24T22:53:48.077 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 6, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 2, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 3, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 1, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:53:48.078 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 2662, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 2651, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 2651, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 11, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-04-24T22:53:48.079 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 55, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 59, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 58, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 58, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 1, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 1, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 1, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 27844522, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 27941110, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2597, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.381379678, 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002842271 2024-04-24T22:53:48.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 35, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 36, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 1, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 20, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 20, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 0, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 42, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 43, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 1, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 4, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 20, 2024-04-24T22:53:48.081 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 16, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 55916, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 231804 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 2030, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 1, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 781, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 34, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 2, 2024-04-24T22:53:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.003527009, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000705401 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 119, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.026905619, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000226097 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000313866, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000313866 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000256882, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000128441 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 12, 2024-04-24T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.002592820, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000216068 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011511208, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001438901 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 12, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.080334115, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006694509 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000349380, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000349380 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-04-24T22:53:48.088 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000954340, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000477170 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.002027652, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001013826 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011452169, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.005726084 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.044661917, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007443652 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-04-24T22:53:48.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 600, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.922678626, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006537797 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-04-24T22:53:48.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.019397813, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002424726 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.092 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000996869, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000996869 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.093 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 5, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 3, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 108, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 54, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 707, 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-04-24T22:53:48.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:48.095 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:48.096 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 315839, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 127, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:48.097 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8032, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 167, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 171587, 2024-04-24T22:53:48.098 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 1264, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-04-24T22:53:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 4406, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 24944895, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 32, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 4406, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4406, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 29.585431057, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006714805 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4406, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 5153 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 4406, 2024-04-24T22:53:48.100 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 1, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 4405, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 329, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 28, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 2598, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 182, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 523, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-04-24T22:53:48.101 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 56, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 599, 2024-04-24T22:53:48.102 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-04-24T22:53:48.103 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 100, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 16, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 825, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 12 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-04-24T22:53:48.104 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 1, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 1, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 1, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 1, 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-04-24T22:53:48.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 9262, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 3772034, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 9262, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 9262, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3772034, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.106 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 4406, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 24945044, 2024-04-24T22:53:48.107 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 4406, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 24945044, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:48.108 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 4406, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 4406, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 4406, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4406, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:48.109 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 1, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 149, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 1, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 149, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x56140d81e500": { 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 2662, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 23746806, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 2662, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2597, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 23746806, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:48.111 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:48.112 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:48.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:48.112 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-24T22:53:48.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:48 smithi171 ceph-mon[32337]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:48.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:48 smithi186 ceph-mon[32753]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:48.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:48 smithi111 ceph-mon[27746]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-24T22:53:48.786 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-24T22:53:48.786 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-24T22:53:48.786 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-24T22:53:48.786 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-04-24T22:53:48.786 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-04-24T22:53:48.786 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T22:53:48.787 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T22:53:49.024 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:53:50.657 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:50 smithi171 ceph-mon[32337]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 251, 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 238, 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1590752, 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 146272, 2024-04-24T22:53:50.657 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.052416765, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015992246, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.038258829, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005999436, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 238, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.013632105, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000057277 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:50.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1590752, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 146272 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 181, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 320, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 187264, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.011368058, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007848658, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001250577, 2024-04-24T22:53:50.659 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 178, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.026739674, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000150222 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 187264 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-24T22:53:50.660 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000317958, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000256122, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:50.661 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:50.662 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:50.663 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:50.664 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 149057, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 32, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:50.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 249, 2024-04-24T22:53:50.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1552915, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 249, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 249, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1552915, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:50.667 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-24T22:53:50.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:50 smithi186 ceph-mon[32753]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:53:50.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:50 smithi111 ceph-mon[27746]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T22:53:51.286 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-24T22:53:51.287 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-24T22:53:51.287 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-24T22:53:51.287 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-24T22:53:51.287 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-24T22:53:51.287 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T22:53:51.287 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T22:53:51.527 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:53:52.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:52 smithi171 ceph-mon[32337]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:52.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:52 smithi186 ceph-mon[32753]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:52.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:52 smithi111 ceph-mon[27746]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:53.205 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-24T22:53:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1928, 2024-04-24T22:53:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1930, 2024-04-24T22:53:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 11724793, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 613114, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.179537659, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.064674876, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.048080466, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.050313878, 2024-04-24T22:53:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1928, 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.075244921, 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000039027 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 362, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 540, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 4285424, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 594914, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.048032268, 2024-04-24T22:53:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015165054, 2024-04-24T22:53:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.015950243, 2024-04-24T22:53:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.012285287, 2024-04-24T22:53:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 537, 2024-04-24T22:53:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.038668349, 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000072008 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 480368 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1171, 2024-04-24T22:53:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1157, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 10231081, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 435147, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.131392093, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.035010322, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.066831468, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.032617106, 2024-04-24T22:53:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1157, 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.037232610, 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000032180 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1617712, 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 149264 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5509, 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.133860079, 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000024298 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-04-24T22:53:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-04-24T22:53:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 32, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 15, 2024-04-24T22:53:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 17, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 12, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-04-24T22:53:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 2, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:53:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-04-24T22:53:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:53:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-04-24T22:53:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-04-24T22:53:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 2662, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 54, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 27844522, 2024-04-24T22:53:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 27941110, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 27929589, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 2662 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 27, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 28, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 1, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 12, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 12, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 0, 2024-04-24T22:53:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 32, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 32, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 53776, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-04-24T22:53:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-04-24T22:53:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-04-24T22:53:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-04-24T22:53:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-04-24T22:53:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-04-24T22:53:53.247 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.248 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.249 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-04-24T22:53:53.250 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-04-24T22:53:53.251 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-04-24T22:53:53.252 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 5, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 3, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 4, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-04-24T22:53:53.253 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 3819, 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:53.254 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:53.255 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:53.256 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:53.257 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:53.258 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 208027, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 88, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:53.259 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8032, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 167, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 133908, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 1006, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-24T22:53:53.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 3187, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 26, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 4, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 3187, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3187, 2024-04-24T22:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.774965761, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000556939 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3187, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3188 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 3187, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 3187, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-04-24T22:53:53.262 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1690, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1494, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 2, 2024-04-24T22:53:53.263 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-04-24T22:53:53.264 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 100, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 15, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 7, 2024-04-24T22:53:53.265 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 7, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-04-24T22:53:53.266 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-04-24T22:53:53.267 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3457, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 25913438, 2024-04-24T22:53:53.268 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3457, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3457, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 25913438, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:53.270 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3187, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 23746816, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3187, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 23746816, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3187, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3187, 2024-04-24T22:53:53.272 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3187, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3187, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:53.273 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-24T22:53:53.274 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x559b8c36a6e0": { 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:53.276 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:53.277 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:53.277 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:53.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:53.277 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-24T22:53:53.835 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-24T22:53:53.836 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-24T22:53:53.836 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-24T22:53:53.836 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-24T22:53:53.836 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T22:53:53.836 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T22:53:53.836 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T22:53:54.073 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T22:53:54.511 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:54 smithi171 ceph-mon[32337]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:53:54.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:54 smithi186 ceph-mon[32753]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:53:54.786 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:54 smithi111 ceph-mon[27746]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:53:55.644 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-24T22:53:55.644 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:55.644 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001001019, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000219624, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:55.645 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:53:55.646 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 186, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3344, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 188240, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.012604995, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.008555727, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001625066, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:53:55.647 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 184, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.025329638, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000137661 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T22:53:55.648 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 188240 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 252, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 241, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1589712, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 148720, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.053914246, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.016238686, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.040033638, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.006326200, 2024-04-24T22:53:55.649 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 240, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.013269481, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055289 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1589712, 2024-04-24T22:53:55.650 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 148720 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:55.651 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:55.652 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:55.653 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:55.654 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148692, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-04-24T22:53:55.655 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-24T22:53:55.656 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 254, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1554202, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:55.657 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 254, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 254, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1554202, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-24T22:53:55.658 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-24T22:53:56.297 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-24T22:53:56.298 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-24T22:53:56.298 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-24T22:53:56.298 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-24T22:53:56.298 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-24T22:53:56.298 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T22:53:56.298 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T22:53:56.550 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:53:56.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:56 smithi171 ceph-mon[32337]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:56.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:56 smithi186 ceph-mon[32753]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:56.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:56 smithi111 ceph-mon[27746]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:58.251 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-24T22:53:58.251 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:53:58.251 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2452, 2024-04-24T22:53:58.251 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2452, 2024-04-24T22:53:58.251 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 698632, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 778919, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.216966131, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.073618215, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.052458783, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.073565419, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2451, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.097042550, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000039593 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:53:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 271, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 259, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1615628, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 150634, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.053631168, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.017155500, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.038839792, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006215229, 2024-04-24T22:53:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 258, 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.015802668, 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000061250 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:58.255 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.256 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.257 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.257 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.257 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1615072, 2024-04-24T22:53:58.258 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 149936 2024-04-24T22:53:58.258 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.258 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:53:58.258 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3, 2024-04-24T22:53:58.258 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 183, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 588, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 482654, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010125330, 2024-04-24T22:53:58.259 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006739585, 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001563739, 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000026044, 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 180, 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.025489611, 2024-04-24T22:53:58.260 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000141608 2024-04-24T22:53:58.261 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.261 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:53:58.261 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.261 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.261 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.262 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.262 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:53:58.262 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 482336 2024-04-24T22:53:58.262 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.262 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:53:58.263 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4175, 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.119576283, 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000028641 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-24T22:53:58.264 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.265 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.266 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-24T22:53:58.267 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-24T22:53:58.268 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-24T22:53:58.269 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-24T22:53:58.269 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-24T22:53:58.270 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-24T22:53:58.271 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-24T22:53:58.272 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:53:58.273 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-24T22:53:58.274 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-04-24T22:53:58.274 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-04-24T22:53:58.274 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-04-24T22:53:58.274 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:53:58.274 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:53:58.275 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:53:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-24T22:53:58.277 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-24T22:53:58.277 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-24T22:53:58.277 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-24T22:53:58.277 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.278 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-24T22:53:58.278 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-24T22:53:58.278 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-24T22:53:58.278 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-24T22:53:58.278 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-24T22:53:58.279 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-24T22:53:58.279 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-24T22:53:58.279 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-24T22:53:58.279 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:53:58.279 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-24T22:53:58.280 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-24T22:53:58.280 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-24T22:53:58.280 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-24T22:53:58.280 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:53:58.281 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-24T22:53:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-24T22:53:58.283 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 5, 2024-04-24T22:53:58.284 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-24T22:53:58.284 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-24T22:53:58.284 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-24T22:53:58.284 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-24T22:53:58.285 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-24T22:53:58.285 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-24T22:53:58.285 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-24T22:53:58.285 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-24T22:53:58.285 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-24T22:53:58.286 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-24T22:53:58.286 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4195815, 2024-04-24T22:53:58.286 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4195674, 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.287 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 5 2024-04-24T22:53:58.288 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.288 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-24T22:53:58.288 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 3, 2024-04-24T22:53:58.288 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 3, 2024-04-24T22:53:58.289 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-24T22:53:58.289 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 1, 2024-04-24T22:53:58.289 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1, 2024-04-24T22:53:58.289 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-04-24T22:53:58.290 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 0, 2024-04-24T22:53:58.290 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 0, 2024-04-24T22:53:58.290 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-24T22:53:58.290 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-24T22:53:58.291 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-24T22:53:58.291 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-24T22:53:58.291 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 49520, 2024-04-24T22:53:58.291 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215420 2024-04-24T22:53:58.292 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.292 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-24T22:53:58.292 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-24T22:53:58.292 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-24T22:53:58.292 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-24T22:53:58.293 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.294 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.295 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.297 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.298 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-24T22:53:58.298 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.298 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.298 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.299 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-24T22:53:58.301 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.303 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.303 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.303 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.303 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-24T22:53:58.304 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.304 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.304 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.304 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.304 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.305 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.306 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.307 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.308 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.309 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.310 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.311 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-24T22:53:58.312 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.314 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-24T22:53:58.315 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.315 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.315 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.315 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.315 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:53:58.316 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-24T22:53:58.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.317 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.317 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.317 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 3811, 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-24T22:53:58.318 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:53:58.319 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:53:58.320 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:53:58.321 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 181331, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 48, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-24T22:53:58.322 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8032, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 167, 2024-04-24T22:53:58.323 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 8464, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 13, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-24T22:53:58.324 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2451, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2451, 2024-04-24T22:53:58.325 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2451, 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.215977780, 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000496114 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-24T22:53:58.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2451, 2024-04-24T22:53:58.327 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2452 2024-04-24T22:53:58.327 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.327 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2451, 2024-04-24T22:53:58.327 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2451, 2024-04-24T22:53:58.327 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1581, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 868, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-24T22:53:58.328 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-24T22:53:58.329 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-24T22:53:58.330 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 15, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 4, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-24T22:53:58.331 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.332 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2722, 2024-04-24T22:53:58.333 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2053080, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2722, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2722, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2053080, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.334 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2451, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1516, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2451, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1516, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2451, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2451, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2451, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2451, 2024-04-24T22:53:58.336 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:53:58.337 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55b1de2606e0": { 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:53:58.339 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:53:58.340 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-24T22:53:58.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:53:58 smithi186 ceph-mon[32753]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:58.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:53:58 smithi171 ceph-mon[32337]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:58.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:53:58 smithi111 ceph-mon[27746]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:53:59.089 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T22:53:59.089 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T22:53:59.089 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-24T22:53:59.089 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-24T22:53:59.089 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T22:53:59.089 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T22:53:59.089 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T22:53:59.333 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:54:00.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:00 smithi171 ceph-mon[32337]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:54:00.698 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:00 smithi186 ceph-mon[32753]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:54:00.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:00 smithi111 ceph-mon[27746]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T22:54:00.927 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 874, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 876, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 232916, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 277996, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.093278776, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.026853764, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.025807747, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.031433007, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 874, 2024-04-24T22:54:00.928 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.038122568, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000043618 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1053, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1041, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1807984, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 399575, 2024-04-24T22:54:00.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.138128047, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.037732644, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.062175620, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.041169130, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1040, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.042094343, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000040475 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.930 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1615344, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 150576 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 869, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1048, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 293312, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 759010, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.116996819, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.039270797, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.029507348, 2024-04-24T22:54:00.931 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.035683693, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1046, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.068974803, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000065941 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 484576 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-24T22:54:00.932 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4202, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.137225796, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000032657 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.933 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-24T22:54:00.934 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-24T22:54:00.935 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-04-24T22:54:00.936 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:54:00.937 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:54:00.938 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:54:00.939 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 12, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-24T22:54:00.940 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4197721, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4197580, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 12 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 3, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 3, 2024-04-24T22:54:00.941 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 1, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 52420, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215420 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-24T22:54:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-24T22:54:00.943 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-24T22:54:00.944 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-24T22:54:00.945 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-24T22:54:00.946 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.947 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.947 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.947 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.947 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.948 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.949 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.950 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.953 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 5, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 4588, 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-24T22:54:00.954 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:54:00.955 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:54:00.956 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 189411, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 52, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:54:00.957 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8032, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 167, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 8464, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 13, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-24T22:54:00.958 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2520, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2520, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2520, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2.187960029, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000868238 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2520, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2521 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.959 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2520, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2520, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1644, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 874, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-24T22:54:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-24T22:54:00.961 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-24T22:54:00.962 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 15, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 5, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 5, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:54:00.963 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:00.964 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2792, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2066092, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2792, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2792, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2066092, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.965 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2520, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 3422, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2520, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 3422, 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:00.966 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2520, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2520, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2520, 2024-04-24T22:54:00.967 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2520, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:00.968 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55af8aae86e0": { 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:00.969 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:00.970 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-24T22:54:01.611 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T22:54:01.611 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T22:54:01.611 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-24T22:54:01.611 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-24T22:54:01.611 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T22:54:01.611 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T22:54:01.612 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T22:54:01.854 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:54:02.656 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:02 smithi171 ceph-mon[32337]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:54:02.667 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:02 smithi186 ceph-mon[32753]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:54:02.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:02 smithi111 ceph-mon[27746]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 349, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 357, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1637502, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 222252, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 14, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.067806731, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.021364533, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.047346350, 2024-04-24T22:54:03.523 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006881465, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 356, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.029673977, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000083353 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 57, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000036258, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000636 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1628208, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 158176 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.524 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 41, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 265, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 10358, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 516633, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 10, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.016594910, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.009043283, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.005109873, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000416890, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 262, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.037268830, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000142247 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.525 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 72, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000028746, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000399 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 5984, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 503664 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 644, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 640, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 168953, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 405309, 2024-04-24T22:54:03.526 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.073054211, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.028016905, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.040377577, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006228795, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 639, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.043758662, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000068479 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1240, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000333628, 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000269 2024-04-24T22:54:03.527 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 4864, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 3792 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 72, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001984607, 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000027563 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.528 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000016480, 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000008240 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-24T22:54:03.529 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 16, 2024-04-24T22:54:03.530 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 9, 2024-04-24T22:54:03.530 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-24T22:54:03.530 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 9, 2024-04-24T22:54:03.530 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002739931, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000304436 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 7, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 11, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 11, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 10, 2024-04-24T22:54:03.531 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 1, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 1, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 1, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 2, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 24, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 16, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 1, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 2, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-24T22:54:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 1, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 5, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 298, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-24T22:54:03.533 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 2 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.534 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 3, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-24T22:54:03.535 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 12, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-24T22:54:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 12, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4197721, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-24T22:54:03.537 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 11, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.018734574, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001703143 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 14, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 14, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 13, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 14, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 11, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 12, 2024-04-24T22:54:03.538 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 1, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 1, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 6, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 5, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 52124, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223612 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 18, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 16, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 36, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.539 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 6, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001512914, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000252152 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.540 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001227017, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000409005 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.541 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.542 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.543 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.544 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.545 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.546 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.547 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.548 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.549 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-24T22:54:03.550 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 7, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 1, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 719, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:54:03.551 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:54:03.552 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 250085, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 103, 2024-04-24T22:54:03.553 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 176, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 2, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8032, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 167, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-24T22:54:03.554 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 56680, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 225, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 50, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 13954, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 33, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 50, 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-24T22:54:03.555 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 50, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.389992927, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.007799858 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 50, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 74 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 50, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 0, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 50, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-04-24T22:54:03.556 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 11, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 8, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-24T22:54:03.557 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-24T22:54:03.558 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 15, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 8, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 8, 2024-04-24T22:54:03.559 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 17, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 2 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.560 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1030, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1704178, 2024-04-24T22:54:03.561 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1030, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1030, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1704178, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:54:03.562 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 50, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 13954, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 50, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 13954, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.563 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 50, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 50, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 50, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 50, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.564 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:03.565 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x561284a32500": { 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 12, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 3417, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 12, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 11, 2024-04-24T22:54:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 3417, 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:03.567 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-24T22:54:04.285 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-24T22:54:04.286 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-24T22:54:04.286 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-24T22:54:04.286 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-24T22:54:04.286 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T22:54:04.286 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T22:54:04.286 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T22:54:04.525 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T22:54:04.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:04 smithi111 ceph-mon[27746]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:04.660 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:04 smithi186 ceph-mon[32753]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:04.661 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:04 smithi171 ceph-mon[32337]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:06.165 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-24T22:54:06.165 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000932311, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000132734, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:06.166 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 187, 2024-04-24T22:54:06.167 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 188320, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012452250, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008753558, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001199264, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 184, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.033226419, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000180578 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:06.168 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 188320 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 260, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 247, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1593824, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 151728, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.052379201, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.016181957, 2024-04-24T22:54:06.169 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.038232807, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006285044, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 247, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.014853565, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000060135 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1593824, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 151728 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.170 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T22:54:06.171 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-24T22:54:06.172 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148709, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 31, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-24T22:54:06.173 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2024-04-24T22:54:06.174 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 258, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1554722, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 258, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-24T22:54:06.175 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 258, 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1554722, 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-24T22:54:06.176 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-24T22:54:06.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:06 smithi186 ceph-mon[32753]: pgmap v379: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:06.769 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T22:54:06.769 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T22:54:06.769 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-24T22:54:06.769 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-24T22:54:06.769 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-24T22:54:06.769 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T22:54:06.781 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T22:54:06.782 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:54:06.782 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:54:06.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:06 smithi111 ceph-mon[27746]: pgmap v379: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:06.842 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-24T22:54:06.842 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:54:06.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:54:06.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:06 smithi171 ceph-mon[32337]: pgmap v379: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 -- 172.21.15.111:0/386436817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50109930 msgr2=0x7fbc50109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 --2- 172.21.15.111:0/386436817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50109930 0x7fbc50109d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009890 tx=0x7fbc4002a360 comp rx=0 tx=0).stop 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 -- 172.21.15.111:0/386436817 shutdown_connections 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 --2- 172.21.15.111:0/386436817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 0x7fbc501107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 --2- 172.21.15.111:0/386436817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50103920 0x7fbc50103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 --2- 172.21.15.111:0/386436817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50109930 0x7fbc50109d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 -- 172.21.15.111:0/386436817 >> 172.21.15.111:0/386436817 conn(0x7fbc500ff3f0 msgr2=0x7fbc50101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 -- 172.21.15.111:0/386436817 shutdown_connections 2024-04-24T22:54:07.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.148+0000 7fbc56600640 1 -- 172.21.15.111:0/386436817 wait complete. 2024-04-24T22:54:07.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.149+0000 7fbc56600640 1 Processor -- start 2024-04-24T22:54:07.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 -- start start 2024-04-24T22:54:07.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 0x7fbc5019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50109930 0x7fbc5019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113d40 con 0x7fbc50109930 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113bc0 con 0x7fbc501042e0 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc56600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113ec0 con 0x7fbc50103920 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc4f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc4fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50109930 0x7fbc5019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc4f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:57862/0 (socket says 172.21.15.111:57862) 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.150+0000 7fbc4f400640 1 -- 172.21.15.111:0/214253629 learned_addr learned my addr 172.21.15.111:0/214253629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4ea00640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 0x7fbc5019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 -- 172.21.15.111:0/214253629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 msgr2=0x7fbc5019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 0x7fbc5019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 -- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50109930 msgr2=0x7fbc5019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50109930 0x7fbc5019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 -- 172.21.15.111:0/214253629 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019f270 con 0x7fbc50103920 2024-04-24T22:54:07.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.151+0000 7fbc4f400640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc40035040 tx=0x7fbc40033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.152+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40037040 con 0x7fbc50103920 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.152+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40040070 con 0x7fbc50103920 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.152+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40003970 con 0x7fbc50103920 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.152+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019f500 con 0x7fbc50103920 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.152+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501a6df0 con 0x7fbc50103920 2024-04-24T22:54:07.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.153+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc50105060 con 0x7fbc50103920 2024-04-24T22:54:07.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.155+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbc400306a0 con 0x7fbc50103920 2024-04-24T22:54:07.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.156+0000 7fbc47e00640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 0x7fbc340a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.157+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fbc400ef230 con 0x7fbc50103920 2024-04-24T22:54:07.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.157+0000 7fbc4ea00640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 0x7fbc340a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.157+0000 7fbc4ea00640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 0x7fbc340a5240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc50101aa0 tx=0x7fbc3c009260 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.161+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc400bbf00 con 0x7fbc50103920 2024-04-24T22:54:07.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.361+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc50068560 con 0x7fbc50103920 2024-04-24T22:54:07.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.363+0000 7fbc47e00640 1 -- 172.21.15.111:0/214253629 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23162 (secure 0 0 0) 0x7fbc400bb800 con 0x7fbc50103920 2024-04-24T22:54:07.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:54:07.365 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":100,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:53:12.235227+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/1711063394":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2723701565":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/564429901":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2298793350":"2024-04-25T22:41:56.483825+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:54:07.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 msgr2=0x7fbc340a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 0x7fbc340a5240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc50101aa0 tx=0x7fbc3c009260 comp rx=0 tx=0).stop 2024-04-24T22:54:07.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 msgr2=0x7fbc5019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc40035040 tx=0x7fbc40033b60 comp rx=0 tx=0).stop 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 shutdown_connections 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc50109930 0x7fbc5019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc501042e0 0x7fbc5019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fbc340a2d80 0x7fbc340a5240 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 --2- 172.21.15.111:0/214253629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50103920 0x7fbc5019a280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 >> 172.21.15.111:0/214253629 conn(0x7fbc500ff3f0 msgr2=0x7fbc500ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 shutdown_connections 2024-04-24T22:54:07.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.367+0000 7fbc56600640 1 -- 172.21.15.111:0/214253629 wait complete. 2024-04-24T22:54:07.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:34:20.135408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:28.597589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:29.595770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:54:07.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:54:07.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 -- 172.21.15.111:0/3080321178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 msgr2=0x7efd5c103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 --2- 172.21.15.111:0/3080321178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c103c70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009930 tx=0x7efd4c02a440 comp rx=0 tx=0).stop 2024-04-24T22:54:07.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 -- 172.21.15.111:0/3080321178 shutdown_connections 2024-04-24T22:54:07.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 --2- 172.21.15.111:0/3080321178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 0x7efd5c1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 --2- 172.21.15.111:0/3080321178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c103c70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 --2- 172.21.15.111:0/3080321178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 0x7efd5c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 -- 172.21.15.111:0/3080321178 >> 172.21.15.111:0/3080321178 conn(0x7efd5c0ff280 msgr2=0x7efd5c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 -- 172.21.15.111:0/3080321178 shutdown_connections 2024-04-24T22:54:07.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.589+0000 7efd63200640 1 -- 172.21.15.111:0/3080321178 wait complete. 2024-04-24T22:54:07.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 Processor -- start 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 -- start start 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 0x7efd5c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 0x7efd5c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c113c10 con 0x7efd5c1037f0 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c113a90 con 0x7efd5c109800 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd63200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c113d90 con 0x7efd5c1041b0 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.590+0000 7efd61e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41574/0 (socket says 172.21.15.111:41574) 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 -- 172.21.15.111:0/213911770 learned_addr learned my addr 172.21.15.111:0/213911770 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61400640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 0x7efd5c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd62800640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 0x7efd5c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 -- 172.21.15.111:0/213911770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 msgr2=0x7efd5c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 0x7efd5c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 -- 172.21.15.111:0/213911770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 msgr2=0x7efd5c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 0x7efd5c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 -- 172.21.15.111:0/213911770 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c19f140 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.591+0000 7efd61e00640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efd5000e9f0 tx=0x7efd5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.592+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5000cd80 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.592+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c19f430 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.592+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd50010910 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.592+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd500188f0 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.592+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a6cc0 con 0x7efd5c1037f0 2024-04-24T22:54:07.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.593+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c0688d0 con 0x7efd5c1037f0 2024-04-24T22:54:07.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.595+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efd50010ac0 con 0x7efd5c1037f0 2024-04-24T22:54:07.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.596+0000 7efd56a00640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 0x7efd440a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.596+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7efd500cf750 con 0x7efd5c1037f0 2024-04-24T22:54:07.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.597+0000 7efd61400640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 0x7efd440a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.598+0000 7efd61400640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 0x7efd440a52c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009900 tx=0x7efd4c0058d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.601+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd5009bcc0 con 0x7efd5c1037f0 2024-04-24T22:54:07.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:07 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/214253629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:07.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:07 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/214253629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:07.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.801+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd5c10bd60 con 0x7efd5c1037f0 2024-04-24T22:54:07.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.801+0000 7efd56a00640 1 -- 172.21.15.111:0/213911770 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7efd5009ffc0 con 0x7efd5c1037f0 2024-04-24T22:54:07.804 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-04-24T22:54:07.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 msgr2=0x7efd440a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 0x7efd440a52c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009900 tx=0x7efd4c0058d0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 msgr2=0x7efd5c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efd5000e9f0 tx=0x7efd5000eeb0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 shutdown_connections 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd5c109800 0x7efd5c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7efd440a2e00 0x7efd440a52c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd5c1041b0 0x7efd5c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 --2- 172.21.15.111:0/213911770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd5c1037f0 0x7efd5c19a150 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 >> 172.21.15.111:0/213911770 conn(0x7efd5c0ff280 msgr2=0x7efd5c0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 shutdown_connections 2024-04-24T22:54:07.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:07.804+0000 7efd63200640 1 -- 172.21.15.111:0/213911770 wait complete. 2024-04-24T22:54:07.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:54:07.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:07 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/214253629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:08.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 -- 172.21.15.111:0/2155162485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 msgr2=0x7f67d8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 --2- 172.21.15.111:0/2155162485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d8109bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 -- 172.21.15.111:0/2155162485 shutdown_connections 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 --2- 172.21.15.111:0/2155162485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81041a0 0x7f67d81106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 --2- 172.21.15.111:0/2155162485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81037e0 0x7f67d8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 --2- 172.21.15.111:0/2155162485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d8109bd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 -- 172.21.15.111:0/2155162485 >> 172.21.15.111:0/2155162485 conn(0x7f67d80ff270 msgr2=0x7f67d8101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:08.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 -- 172.21.15.111:0/2155162485 shutdown_connections 2024-04-24T22:54:08.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.035+0000 7f67dee00640 1 -- 172.21.15.111:0/2155162485 wait complete. 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.036+0000 7f67dee00640 1 Processor -- start 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.036+0000 7f67dee00640 1 -- start start 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 0x7f67d819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8113c50 con 0x7f67d81037e0 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8113ad0 con 0x7f67d81097f0 2024-04-24T22:54:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dee00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8113dd0 con 0x7f67d81041a0 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dda00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dda00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41596/0 (socket says 172.21.15.111:41596) 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dda00640 1 -- 172.21.15.111:0/655025989 learned_addr learned my addr 172.21.15.111:0/655025989 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 0x7f67d819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67de400640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 -- 172.21.15.111:0/655025989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 msgr2=0x7f67d819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 -- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 msgr2=0x7f67d819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dd000640 1 -- 172.21.15.111:0/655025989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d819f180 con 0x7f67d81041a0 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.037+0000 7f67dda00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67dd000640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 0x7f67d819a6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ccc0 con 0x7f67d81041a0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d819f470 con 0x7f67d81041a0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c4005020 con 0x7f67d81041a0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40049e0 con 0x7f67d81041a0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.038+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a6d00 con 0x7f67d81041a0 2024-04-24T22:54:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.039+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d8067bf0 con 0x7f67d81041a0 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.042+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f67c4005560 con 0x7f67d81041a0 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.043+0000 7f67d6000640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 0x7f67c00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.044+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f67c40cf470 con 0x7f67d81041a0 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.044+0000 7f67dda00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 0x7f67c00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.044+0000 7f67dda00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 0x7f67c00a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000c00 tx=0x7f67cc0046c0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.045+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67c40051d0 con 0x7f67d81041a0 2024-04-24T22:54:08.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.244+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67d810bd50 con 0x7f67d81041a0 2024-04-24T22:54:08.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.245+0000 7f67d6000640 1 -- 172.21.15.111:0/655025989 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f67c409c0c0 con 0x7f67d81041a0 2024-04-24T22:54:08.247 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-04-24T22:54:08.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 msgr2=0x7f67c00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 0x7f67c00a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000c00 tx=0x7f67cc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 msgr2=0x7f67d819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 0x7f67d819a6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 shutdown_connections 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d81097f0 0x7f67d819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f67c00a2d00 0x7f67c00a51c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67d81041a0 0x7f67d819a6d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 --2- 172.21.15.111:0/655025989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67d81037e0 0x7f67d819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 >> 172.21.15.111:0/655025989 conn(0x7f67d80ff270 msgr2=0x7f67d80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 shutdown_connections 2024-04-24T22:54:08.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.248+0000 7f67dee00640 1 -- 172.21.15.111:0/655025989 wait complete. 2024-04-24T22:54:08.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.475+0000 7f4da7000640 1 -- 172.21.15.111:0/718272611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da01015e0 msgr2=0x7f4da0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.475+0000 7f4da7000640 1 --2- 172.21.15.111:0/718272611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da01015e0 0x7f4da0101a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009cd0 tx=0x7f4d8c02a8e0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 -- 172.21.15.111:0/718272611 shutdown_connections 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 --2- 172.21.15.111:0/718272611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0101fa0 0x7f4da010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 --2- 172.21.15.111:0/718272611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da01015e0 0x7f4da0101a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 --2- 172.21.15.111:0/718272611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da0107680 0x7f4da0107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 -- 172.21.15.111:0/718272611 >> 172.21.15.111:0/718272611 conn(0x7f4da00fd110 msgr2=0x7f4da00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 -- 172.21.15.111:0/718272611 shutdown_connections 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.476+0000 7f4da7000640 1 -- 172.21.15.111:0/718272611 wait complete. 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 Processor -- start 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 -- start start 2024-04-24T22:54:08.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 0x7f4da0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da0101fa0 0x7f4da01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01119c0 con 0x7f4da0107680 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111840 con 0x7f4da01015e0 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.477+0000 7f4da7000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111b40 con 0x7f4da0101fa0 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41618/0 (socket says 172.21.15.111:41618) 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 -- 172.21.15.111:0/1890851103 learned_addr learned my addr 172.21.15.111:0/1890851103 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da5c00640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 0x7f4da0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da5200640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da0101fa0 0x7f4da01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 -- 172.21.15.111:0/1890851103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da0101fa0 msgr2=0x7f4da01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da0101fa0 0x7f4da01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 -- 172.21.15.111:0/1890851103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 msgr2=0x7f4da0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 0x7f4da0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 -- 172.21.15.111:0/1890851103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da019ce70 con 0x7f4da0107680 2024-04-24T22:54:08.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da5c00640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 0x7f4da0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:08.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.478+0000 7f4da6600640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e970 tx=0x7f4d9800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.479+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800ccc0 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.479+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da019d160 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.479+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9800ce60 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.479+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98010490 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.479+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01a49f0 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.480+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0068480 con 0x7f4da0107680 2024-04-24T22:54:08.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.482+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4d98005020 con 0x7f4da0107680 2024-04-24T22:54:08.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.483+0000 7f4d9e000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 0x7f4d840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.483+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f4d980cf920 con 0x7f4da0107680 2024-04-24T22:54:08.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.484+0000 7f4da5c00640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 0x7f4d840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.484+0000 7f4da5c00640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 0x7f4d840a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4da0102720 tx=0x7f4d90005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.487+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d9809c4e0 con 0x7f4da0107680 2024-04-24T22:54:08.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:08 smithi111 ceph-mon[27746]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:08 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/213911770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:08 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/655025989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.684+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4da0109b50 con 0x7f4da0107680 2024-04-24T22:54:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.685+0000 7f4d9e000640 1 -- 172.21.15.111:0/1890851103 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f4d9809bde0 con 0x7f4da0107680 2024-04-24T22:54:08.687 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-04-24T22:54:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 msgr2=0x7f4d840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 0x7f4d840a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4da0102720 tx=0x7f4d90005f90 comp rx=0 tx=0).stop 2024-04-24T22:54:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 msgr2=0x7f4da019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e970 tx=0x7f4d9800ed70 comp rx=0 tx=0).stop 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 shutdown_connections 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da0107680 0x7f4da019c740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f4d840a2d00 0x7f4d840a51c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da0101fa0 0x7f4da01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 --2- 172.21.15.111:0/1890851103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4da01015e0 0x7f4da0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 >> 172.21.15.111:0/1890851103 conn(0x7f4da00fd110 msgr2=0x7f4da00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 shutdown_connections 2024-04-24T22:54:08.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.688+0000 7f4da7000640 1 -- 172.21.15.111:0/1890851103 wait complete. 2024-04-24T22:54:08.710 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:54:08.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:08 smithi186 ceph-mon[32753]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:08.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:08 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/213911770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:08 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/655025989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:08 smithi171 ceph-mon[32337]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:08 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/213911770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:08 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/655025989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T22:54:08.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 -- 172.21.15.111:0/3208925516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 msgr2=0x7fad9c073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 --2- 172.21.15.111:0/3208925516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c073f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009900 tx=0x7fad8c02a480 comp rx=0 tx=0).stop 2024-04-24T22:54:08.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 -- 172.21.15.111:0/3208925516 shutdown_connections 2024-04-24T22:54:08.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 --2- 172.21.15.111:0/3208925516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 0x7fad9c110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 --2- 172.21.15.111:0/3208925516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 --2- 172.21.15.111:0/3208925516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c073f50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.907+0000 7fada0e00640 1 -- 172.21.15.111:0/3208925516 >> 172.21.15.111:0/3208925516 conn(0x7fad9c0fd1c0 msgr2=0x7fad9c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:08.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.908+0000 7fada0e00640 1 -- 172.21.15.111:0/3208925516 shutdown_connections 2024-04-24T22:54:08.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.908+0000 7fada0e00640 1 -- 172.21.15.111:0/3208925516 wait complete. 2024-04-24T22:54:08.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 Processor -- start 2024-04-24T22:54:08.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 -- start start 2024-04-24T22:54:08.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 0x7fad9c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c113d20 con 0x7fad9c074490 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c113ba0 con 0x7fad9c073b70 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fada0e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c113ea0 con 0x7fad9c10df30 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fad91400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fad91400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41632/0 (socket says 172.21.15.111:41632) 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.909+0000 7fad91400640 1 -- 172.21.15.111:0/1055206806 learned_addr learned my addr 172.21.15.111:0/1055206806 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad9a800640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 0x7fad9c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad99e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 -- 172.21.15.111:0/1055206806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 msgr2=0x7fad9c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 0x7fad9c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 -- 172.21.15.111:0/1055206806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 msgr2=0x7fad9c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 -- 172.21.15.111:0/1055206806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c19f2d0 con 0x7fad9c074490 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad91400640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.910+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8400cd80 con 0x7fad9c074490 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.911+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c19f5c0 con 0x7fad9c074490 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.911+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad84010910 con 0x7fad9c074490 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.911+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84018990 con 0x7fad9c074490 2024-04-24T22:54:08.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.911+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1a6e50 con 0x7fad9c074490 2024-04-24T22:54:08.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.912+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9c067cd0 con 0x7fad9c074490 2024-04-24T22:54:08.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.914+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fad84010ac0 con 0x7fad9c074490 2024-04-24T22:54:08.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.915+0000 7fad93400640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 0x7fad800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:08.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.915+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fad840cf7a0 con 0x7fad9c074490 2024-04-24T22:54:08.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.916+0000 7fad99e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 0x7fad800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:08.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.916+0000 7fad99e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 0x7fad800a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c005b50 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:08.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:08.920+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad8409bd10 con 0x7fad9c074490 2024-04-24T22:54:09.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.133+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fad9c109110 con 0x7fad9c074490 2024-04-24T22:54:09.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.134+0000 7fad93400640 1 -- 172.21.15.111:0/1055206806 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+960 (secure 0 0 0) 0x7fad840a0010 con 0x7fad9c074490 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6801/1872264645 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/1711063394 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2723701565 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/564429901 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2026940770 2024-04-25T22:31:27.425198+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/1436625216 2024-04-25T22:31:27.425198+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6800/1872264645 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2298793350 2024-04-25T22:41:56.483825+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6801/3447332391 2024-04-25T22:31:27.425198+0000 2024-04-24T22:54:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6800/2142544018 2024-04-25T22:30:53.108811+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/1327559640 2024-04-25T22:30:53.108811+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/4190678364 2024-04-25T22:30:53.108811+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/449834621 2024-04-25T22:31:27.425198+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6801/2142544018 2024-04-25T22:30:53.108811+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/4240227320 2024-04-25T22:30:53.108811+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6800/3447332391 2024-04-25T22:31:27.425198+0000 2024-04-24T22:54:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:listed 16 entries 2024-04-24T22:54:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 msgr2=0x7fad800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 0x7fad800a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c005b50 comp rx=0 tx=0).stop 2024-04-24T22:54:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 msgr2=0x7fad9c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400eeb0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 shutdown_connections 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad800a2d00 0x7fad800a51c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad9c10df30 0x7fad9c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c074490 0x7fad9c19a870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 --2- 172.21.15.111:0/1055206806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad9c073b70 0x7fad9c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 >> 172.21.15.111:0/1055206806 conn(0x7fad9c0fd1c0 msgr2=0x7fad9c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 shutdown_connections 2024-04-24T22:54:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.137+0000 7fada0e00640 1 -- 172.21.15.111:0/1055206806 wait complete. 2024-04-24T22:54:09.157 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T22:54:09.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:54:09.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T22:54:09.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:54:09.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:54:09.264 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T22:54:09.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:54:09.327 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-04-24T22:54:09.366 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T22:54:09.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:54:09.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:54:09.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:54:09.474 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-04-24T22:54:09.575 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T22:54:09.586 INFO:tasks.cephadm:Teardown begin 2024-04-24T22:54:09.587 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T22:54:09.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:54:09.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:09 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1890851103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:54:09.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:09 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1055206806' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:54:09.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 -- 172.21.15.111:0/1146365385 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c075c70 msgr2=0x7f862c076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:09.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 --2- 172.21.15.111:0/1146365385 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c075c70 0x7f862c076050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f861c009890 tx=0x7f861c02a360 comp rx=0 tx=0).stop 2024-04-24T22:54:09.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 -- 172.21.15.111:0/1146365385 shutdown_connections 2024-04-24T22:54:09.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 --2- 172.21.15.111:0/1146365385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 --2- 172.21.15.111:0/1146365385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c076590 0x7f862c112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 --2- 172.21.15.111:0/1146365385 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c075c70 0x7f862c076050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.773+0000 7f8631800640 1 -- 172.21.15.111:0/1146365385 >> 172.21.15.111:0/1146365385 conn(0x7f862c0ff3f0 msgr2=0x7f862c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.774+0000 7f8631800640 1 -- 172.21.15.111:0/1146365385 shutdown_connections 2024-04-24T22:54:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.774+0000 7f8631800640 1 -- 172.21.15.111:0/1146365385 wait complete. 2024-04-24T22:54:09.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 Processor -- start 2024-04-24T22:54:09.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 -- start start 2024-04-24T22:54:09.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c075c70 0x7f862c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 0x7f862c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f862c118130 con 0x7f862c1125c0 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f862c117fb0 con 0x7f862c076590 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.775+0000 7f8631800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1182b0 con 0x7f862c075c70 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f8623e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 0x7f862c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41652/0 (socket says 172.21.15.111:41652) 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 -- 172.21.15.111:0/1090242663 learned_addr learned my addr 172.21.15.111:0/1090242663 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862aa00640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c075c70 0x7f862c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 -- 172.21.15.111:0/1090242663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c075c70 msgr2=0x7f862c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c075c70 0x7f862c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 -- 172.21.15.111:0/1090242663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 msgr2=0x7f862c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 0x7f862c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 -- 172.21.15.111:0/1090242663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c1a3650 con 0x7f862c1125c0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f8623e00640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 0x7f862c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.776+0000 7f862b400640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f862400c3f0 tx=0x7f862400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.777+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624003970 con 0x7f862c1125c0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.777+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c1a3940 con 0x7f862c1125c0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.777+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862400b690 con 0x7f862c1125c0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.777+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624004420 con 0x7f862c1125c0 2024-04-24T22:54:09.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.777+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c1ab1d0 con 0x7f862c1125c0 2024-04-24T22:54:09.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.778+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c068560 con 0x7f862c1125c0 2024-04-24T22:54:09.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.780+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f862401d020 con 0x7f862c1125c0 2024-04-24T22:54:09.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:09 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1890851103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:54:09.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:09 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1055206806' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:54:09.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.782+0000 7f8623400640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 0x7f86100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:09.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.782+0000 7f862aa00640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 0x7f86100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:09.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.782+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f86240cf220 con 0x7f862c1125c0 2024-04-24T22:54:09.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.783+0000 7f862aa00640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 0x7f86100a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f861c035040 tx=0x7f861c0056d0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:09.786+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f862409be40 con 0x7f862c1125c0 2024-04-24T22:54:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:09 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1890851103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T22:54:09.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:09 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1055206806' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:54:10.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.033+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f862c10d7b0 con 0x7f862c1125c0 2024-04-24T22:54:10.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.035+0000 7f8623400640 1 -- 172.21.15.111:0/1090242663 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11877 (secure 0 0 0) 0x7f862409b740 con 0x7f862c1125c0 2024-04-24T22:54:10.036 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:54:10.037 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:41:56.218689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24587,24556,24574]},"id":1}]} 2024-04-24T22:54:10.037 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 24 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 msgr2=0x7f86100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 0x7f86100a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f861c035040 tx=0x7f861c0056d0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 msgr2=0x7f862c1a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f862400c3f0 tx=0x7f862400c8b0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 shutdown_connections 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c1125c0 0x7f862c1a2f20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f862c076590 0x7f862c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f86100a2d00 0x7f86100a51c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 --2- 172.21.15.111:0/1090242663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c075c70 0x7f862c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 >> 172.21.15.111:0/1090242663 conn(0x7f862c0ff3f0 msgr2=0x7f862c113c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:10.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 shutdown_connections 2024-04-24T22:54:10.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.038+0000 7f8631800640 1 -- 172.21.15.111:0/1090242663 wait complete. 2024-04-24T22:54:10.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 -- 172.21.15.111:0/3603924744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 msgr2=0x7fb7f8076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 --2- 172.21.15.111:0/3603924744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f8076050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009890 tx=0x7fb7e802a360 comp rx=0 tx=0).stop 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 -- 172.21.15.111:0/3603924744 shutdown_connections 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 --2- 172.21.15.111:0/3603924744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 --2- 172.21.15.111:0/3603924744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f8112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 --2- 172.21.15.111:0/3603924744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f8076050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 -- 172.21.15.111:0/3603924744 >> 172.21.15.111:0/3603924744 conn(0x7fb7f80ff3f0 msgr2=0x7fb7f8101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 -- 172.21.15.111:0/3603924744 shutdown_connections 2024-04-24T22:54:10.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.260+0000 7fb7fec00640 1 -- 172.21.15.111:0/3603924744 wait complete. 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.261+0000 7fb7fec00640 1 Processor -- start 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.261+0000 7fb7fec00640 1 -- start start 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.261+0000 7fb7fec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.261+0000 7fb7fec00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fec00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fec00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8118130 con 0x7fb7f81125c0 2024-04-24T22:54:10.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8117fb0 con 0x7fb7f8075c70 2024-04-24T22:54:10.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fec00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81182b0 con 0x7fb7f8076590 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fcc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7efe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fcc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41682/0 (socket says 172.21.15.111:41682) 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.111:47852/0 (socket says 172.21.15.111:47852) 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7fcc00640 1 -- 172.21.15.111:0/2196216093 learned_addr learned my addr 172.21.15.111:0/2196216093 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 -- 172.21.15.111:0/2196216093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 msgr2=0x7fb7f819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 -- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 msgr2=0x7fb7f81a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7f7e00640 1 -- 172.21.15.111:0/2196216093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f81a3660 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.262+0000 7fb7efe00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7f7e00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009780 tx=0x7fb7e8033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8037040 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81a38f0 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e8033d90 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8003970 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.263+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81ab1e0 con 0x7fb7f8075c70 2024-04-24T22:54:10.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.264+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8068560 con 0x7fb7f8075c70 2024-04-24T22:54:10.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.266+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb7e8042020 con 0x7fb7f8075c70 2024-04-24T22:54:10.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.268+0000 7fb7f5600640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 0x7fb7d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.268+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fb7e80eeec0 con 0x7fb7f8075c70 2024-04-24T22:54:10.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.268+0000 7fb7efe00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 0x7fb7d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.269+0000 7fb7efe00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 0x7fb7d80a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8101830 tx=0x7fb7e0005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:10.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.270+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7e8049020 con 0x7fb7f8075c70 2024-04-24T22:54:10.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.512+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7f819f9c0 con 0x7fb7f8075c70 2024-04-24T22:54:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.513+0000 7fb7f5600640 1 -- 172.21.15.111:0/2196216093 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11877 (secure 0 0 0) 0x7fb7f819f9c0 con 0x7fb7f8075c70 2024-04-24T22:54:10.515 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:54:10.515 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/425814781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":425814781},{"type":"v1","addr":"172.21.15.111:6835","nonce":425814781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2737045039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2737045039},{"type":"v1","addr":"172.21.15.111:6837","nonce":2737045039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/667255252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":667255252},{"type":"v1","addr":"172.21.15.111:6841","nonce":667255252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1686750740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1686750740},{"type":"v1","addr":"172.21.15.186:6833","nonce":1686750740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/254707808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":254707808},{"type":"v1","addr":"172.21.15.171:6833","nonce":254707808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4258808378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4258808378},{"type":"v1","addr":"172.21.15.171:6835","nonce":4258808378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T22:39:30.113682+0000","modified":"2024-04-24T22:41:56.218689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24574,"mds_2":24556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/964018737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":964018737},{"type":"v1","addr":"172.21.15.186:6835","nonce":964018737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1876976469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1876976469},{"type":"v1","addr":"172.21.15.186:6837","nonce":1876976469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3780798137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3780798137},{"type":"v1","addr":"172.21.15.171:6837","nonce":3780798137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2246377551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2246377551},{"type":"v1","addr":"172.21.15.186:6839","nonce":2246377551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/84522869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":84522869},{"type":"v1","addr":"172.21.15.111:6839","nonce":84522869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6839/3821671666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3821671666},{"type":"v1","addr":"172.21.15.171:6839","nonce":3821671666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24587,"qdb_cluster":[24587,24556,24574]},"id":1}]} 2024-04-24T22:54:10.515 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 24 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 msgr2=0x7fb7d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 0x7fb7d80a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8101830 tx=0x7fb7e0005f90 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 msgr2=0x7fb7f819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009780 tx=0x7fb7e8033ae0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 shutdown_connections 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f81125c0 0x7fb7f81a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fb7d80a2d00 0x7fb7d80a51c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8076590 0x7fb7f819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 --2- 172.21.15.111:0/2196216093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7f8075c70 0x7fb7f819e670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 >> 172.21.15.111:0/2196216093 conn(0x7fb7f80ff3f0 msgr2=0x7fb7f8113cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 shutdown_connections 2024-04-24T22:54:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.516+0000 7fb7fec00640 1 -- 172.21.15.111:0/2196216093 wait complete. 2024-04-24T22:54:10.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:54:10.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:10 smithi186 ceph-mon[32753]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:10.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:10 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1090242663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:10.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 -- 172.21.15.111:0/2709891709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 msgr2=0x7fe2581083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 --2- 172.21.15.111:0/2709891709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe2581083a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009900 tx=0x7fe24c02a3f0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 -- 172.21.15.111:0/2709891709 shutdown_connections 2024-04-24T22:54:10.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 --2- 172.21.15.111:0/2709891709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe25810d090 0x7fe258114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 --2- 172.21.15.111:0/2709891709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe258108970 0x7fe25810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 --2- 172.21.15.111:0/2709891709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe2581083a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 -- 172.21.15.111:0/2709891709 >> 172.21.15.111:0/2709891709 conn(0x7fe258074f40 msgr2=0x7fe258106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:10.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 -- 172.21.15.111:0/2709891709 shutdown_connections 2024-04-24T22:54:10.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.748+0000 7fe25f800640 1 -- 172.21.15.111:0/2709891709 wait complete. 2024-04-24T22:54:10.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 Processor -- start 2024-04-24T22:54:10.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 -- start start 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe25810d090 0x7fe25819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 0x7fe2581a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe258118190 con 0x7fe258107fc0 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe258118010 con 0x7fe2581a2ee0 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.749+0000 7fe25f800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe258118310 con 0x7fe25810d090 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41700/0 (socket says 172.21.15.111:41700) 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 -- 172.21.15.111:0/770865654 learned_addr learned my addr 172.21.15.111:0/770865654 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe257e00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe25810d090 0x7fe25819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25d800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 0x7fe2581a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 -- 172.21.15.111:0/770865654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe25810d090 msgr2=0x7fe25819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe25810d090 0x7fe25819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 -- 172.21.15.111:0/770865654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 msgr2=0x7fe2581a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 0x7fe2581a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 -- 172.21.15.111:0/770865654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2581a39d0 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25d800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 0x7fe2581a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.750+0000 7fe25ce00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe24c004770 tx=0x7fe24c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.751+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c03f070 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.751+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c031ca0 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.751+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c035030 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.751+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2581a3c60 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.751+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2581ab480 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.752+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2580689b0 con 0x7fe258107fc0 2024-04-24T22:54:10.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.754+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe24c042070 con 0x7fe258107fc0 2024-04-24T22:54:10.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.755+0000 7fe255600640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 0x7fe2400a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:10.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.755+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fe24c0efaa0 con 0x7fe258107fc0 2024-04-24T22:54:10.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.756+0000 7fe257e00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 0x7fe2400a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:10.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.756+0000 7fe257e00640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 0x7fe2400a5240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe258107660 tx=0x7fe248005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:10.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.759+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe24c0bc720 con 0x7fe258107fc0 2024-04-24T22:54:10.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:10 smithi111 ceph-mon[27746]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:10.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:10 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1090242663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:10 smithi171 ceph-mon[32337]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T22:54:10.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:10 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1090242663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:10.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.961+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2581102c0 con 0x7fe258107fc0 2024-04-24T22:54:10.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.965+0000 7fe255600640 1 -- 172.21.15.111:0/770865654 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23162 (secure 0 0 0) 0x7fe24c0bc020 con 0x7fe258107fc0 2024-04-24T22:54:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-24T22:54:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":100,"fsid":"0d3ba9b8-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:30:20.053310+0000","modified":"2024-04-24T22:53:12.235227+0000","last_up_change":"2024-04-24T22:38:32.196033+0000","last_in_change":"2024-04-24T22:38:11.384519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:34:20.135408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:28.597589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:29.595770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3aa0e2-3c78-4deb-9ab1-e1cc5c981fa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6803","nonce":1011608049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6805","nonce":1011608049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6809","nonce":1011608049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1011608049},{"type":"v1","addr":"172.21.15.111:6807","nonce":1011608049}]},"public_addr":"172.21.15.111:6803/1011608049","cluster_addr":"172.21.15.111:6805/1011608049","heartbeat_back_addr":"172.21.15.111:6809/1011608049","heartbeat_front_addr":"172.21.15.111:6807/1011608049","state":["exists","up"]},{"osd":1,"uuid":"9336b8fd-028c-49e6-ac02-839e7eb43edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6801","nonce":1334857356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6803","nonce":1334857356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6807","nonce":1334857356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1334857356},{"type":"v1","addr":"172.21.15.171:6805","nonce":1334857356}]},"public_addr":"172.21.15.171:6801/1334857356","cluster_addr":"172.21.15.171:6803/1334857356","heartbeat_back_addr":"172.21.15.171:6807/1334857356","heartbeat_front_addr":"172.21.15.171:6805/1334857356","state":["exists","up"]},{"osd":2,"uuid":"43993a4c-fb63-47d7-af12-b6cc7f7c2d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6801","nonce":2871610305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6803","nonce":2871610305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6807","nonce":2871610305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2871610305},{"type":"v1","addr":"172.21.15.186:6805","nonce":2871610305}]},"public_addr":"172.21.15.186:6801/2871610305","cluster_addr":"172.21.15.186:6803/2871610305","heartbeat_back_addr":"172.21.15.186:6807/2871610305","heartbeat_front_addr":"172.21.15.186:6805/2871610305","state":["exists","up"]},{"osd":3,"uuid":"4d7cb3cc-81a9-4f99-802b-d29c399eb6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6811","nonce":3102831357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6813","nonce":3102831357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6817","nonce":3102831357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3102831357},{"type":"v1","addr":"172.21.15.111:6815","nonce":3102831357}]},"public_addr":"172.21.15.111:6811/3102831357","cluster_addr":"172.21.15.111:6813/3102831357","heartbeat_back_addr":"172.21.15.111:6817/3102831357","heartbeat_front_addr":"172.21.15.111:6815/3102831357","state":["exists","up"]},{"osd":4,"uuid":"7d8ef87e-1192-4e17-8861-c2214d514b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6809","nonce":2305118347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6811","nonce":2305118347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6815","nonce":2305118347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2305118347},{"type":"v1","addr":"172.21.15.171:6813","nonce":2305118347}]},"public_addr":"172.21.15.171:6809/2305118347","cluster_addr":"172.21.15.171:6811/2305118347","heartbeat_back_addr":"172.21.15.171:6815/2305118347","heartbeat_front_addr":"172.21.15.171:6813/2305118347","state":["exists","up"]},{"osd":5,"uuid":"1ac00518-3f56-44e1-84fb-9f5f57fb12a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6809","nonce":2042071677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6811","nonce":2042071677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6815","nonce":2042071677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2042071677},{"type":"v1","addr":"172.21.15.186:6813","nonce":2042071677}]},"public_addr":"172.21.15.186:6809/2042071677","cluster_addr":"172.21.15.186:6811/2042071677","heartbeat_back_addr":"172.21.15.186:6815/2042071677","heartbeat_front_addr":"172.21.15.186:6813/2042071677","state":["exists","up"]},{"osd":6,"uuid":"e612d700-d87e-4031-b2f9-6455030a282b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6819","nonce":525582463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6821","nonce":525582463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6825","nonce":525582463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":525582463},{"type":"v1","addr":"172.21.15.111:6823","nonce":525582463}]},"public_addr":"172.21.15.111:6819/525582463","cluster_addr":"172.21.15.111:6821/525582463","heartbeat_back_addr":"172.21.15.111:6825/525582463","heartbeat_front_addr":"172.21.15.111:6823/525582463","state":["exists","up"]},{"osd":7,"uuid":"ac3e009c-2c29-480c-b226-9df2d6f0d5ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6817","nonce":2409942140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6819","nonce":2409942140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6823","nonce":2409942140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2409942140},{"type":"v1","addr":"172.21.15.171:6821","nonce":2409942140}]},"public_addr":"172.21.15.171:6817/2409942140","cluster_addr":"172.21.15.171:6819/2409942140","heartbeat_back_addr":"172.21.15.171:6823/2409942140","heartbeat_front_addr":"172.21.15.171:6821/2409942140","state":["exists","up"]},{"osd":8,"uuid":"789c1edf-d22f-4a42-9b25-02cedbccadd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6817","nonce":796165885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6819","nonce":796165885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6823","nonce":796165885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":796165885},{"type":"v1","addr":"172.21.15.186:6821","nonce":796165885}]},"public_addr":"172.21.15.186:6817/796165885","cluster_addr":"172.21.15.186:6819/796165885","heartbeat_back_addr":"172.21.15.186:6823/796165885","heartbeat_front_addr":"172.21.15.186:6821/796165885","state":["exists","up"]},{"osd":9,"uuid":"aa3d5a06-570a-4e45-874e-c64a64f66b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6827","nonce":637881617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6829","nonce":637881617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6833","nonce":637881617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":637881617},{"type":"v1","addr":"172.21.15.111:6831","nonce":637881617}]},"public_addr":"172.21.15.111:6827/637881617","cluster_addr":"172.21.15.111:6829/637881617","heartbeat_back_addr":"172.21.15.111:6833/637881617","heartbeat_front_addr":"172.21.15.111:6831/637881617","state":["exists","up"]},{"osd":10,"uuid":"13df39cd-031b-431a-90a9-76d1c09f1922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6825","nonce":4154918762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6827","nonce":4154918762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6831","nonce":4154918762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4154918762},{"type":"v1","addr":"172.21.15.171:6829","nonce":4154918762}]},"public_addr":"172.21.15.171:6825/4154918762","cluster_addr":"172.21.15.171:6827/4154918762","heartbeat_back_addr":"172.21.15.171:6831/4154918762","heartbeat_front_addr":"172.21.15.171:6829/4154918762","state":["exists","up"]},{"osd":11,"uuid":"add0e95d-86a9-4c00-8c5e-e775d9afabbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6825","nonce":33009577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6827","nonce":33009577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6831","nonce":33009577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":33009577},{"type":"v1","addr":"172.21.15.186:6829","nonce":33009577}]},"public_addr":"172.21.15.186:6825/33009577","cluster_addr":"172.21.15.186:6827/33009577","heartbeat_back_addr":"172.21.15.186:6831/33009577","heartbeat_front_addr":"172.21.15.186:6829/33009577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:25.428641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:33:50.538346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:15.708211+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:34:48.547176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:14.861248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:35:39.846852+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:13.612246+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:36:39.848850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:04.803455+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:37:38.267891+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:04.619664+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:29.359907+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/1711063394":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2723701565":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/564429901":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2026940770":"2024-04-25T22:31:27.425198+0000","172.21.15.111:0/1436625216":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/1872264645":"2024-04-25T22:41:56.483825+0000","172.21.15.111:0/2298793350":"2024-04-25T22:41:56.483825+0000","172.21.15.111:6801/3447332391":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6800/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/1327559640":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4190678364":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/449834621":"2024-04-25T22:31:27.425198+0000","172.21.15.111:6801/2142544018":"2024-04-25T22:30:53.108811+0000","172.21.15.111:0/4240227320":"2024-04-25T22:30:53.108811+0000","172.21.15.111:6800/3447332391":"2024-04-25T22:31:27.425198+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 msgr2=0x7fe2400a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 0x7fe2400a5240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe258107660 tx=0x7fe248005f90 comp rx=0 tx=0).stop 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 msgr2=0x7fe25819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe24c004770 tx=0x7fe24c003c10 comp rx=0 tx=0).stop 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 shutdown_connections 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2581a2ee0 0x7fe2581a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fe2400a2d80 0x7fe2400a5240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe25810d090 0x7fe25819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 --2- 172.21.15.111:0/770865654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe258107fc0 0x7fe25819e670 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:10.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 >> 172.21.15.111:0/770865654 conn(0x7fe258074f40 msgr2=0x7fe2580755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:10.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 shutdown_connections 2024-04-24T22:54:10.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:10.968+0000 7fe25f800640 1 -- 172.21.15.111:0/770865654 wait complete. 2024-04-24T22:54:10.988 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 -- 172.21.15.111:0/695859893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 msgr2=0x7f240c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 --2- 172.21.15.111:0/695859893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c108140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009900 tx=0x7f23f402a480 comp rx=0 tx=0).stop 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 -- 172.21.15.111:0/695859893 shutdown_connections 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 --2- 172.21.15.111:0/695859893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c114aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 --2- 172.21.15.111:0/695859893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c108140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 --2- 172.21.15.111:0/695859893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 0x7f240c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 -- 172.21.15.111:0/695859893 >> 172.21.15.111:0/695859893 conn(0x7f240c077560 msgr2=0x7f240c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 -- 172.21.15.111:0/695859893 shutdown_connections 2024-04-24T22:54:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.189+0000 7f2411a00640 1 -- 172.21.15.111:0/695859893 wait complete. 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.190+0000 7f2411a00640 1 Processor -- start 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.190+0000 7f2411a00640 1 -- start start 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.190+0000 7f2411a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.190+0000 7f2411a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.190+0000 7f2411a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 0x7f240c1a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2411a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f240c118010 con 0x7f240c107ce0 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2411a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f240c117e90 con 0x7f240c10dd00 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2411a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f240c118190 con 0x7f240c108680 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f240be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f240be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41712/0 (socket says 172.21.15.111:41712) 2024-04-24T22:54:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f240be00640 1 -- 172.21.15.111:0/1830554681 learned_addr learned my addr 172.21.15.111:0/1830554681 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2411000640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 0x7f240c1a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 -- 172.21.15.111:0/1830554681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 msgr2=0x7f240c1a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 0x7f240c1a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 -- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 msgr2=0x7f240c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f2403400640 1 -- 172.21.15.111:0/1830554681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c1a33f0 con 0x7f240c108680 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.191+0000 7f240be00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2403400640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c19e990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23f402a9a0 tx=0x7f23f4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f403f070 con 0x7f240c108680 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23f4031c90 con 0x7f240c108680 2024-04-24T22:54:11.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c1a3680 con 0x7f240c108680 2024-04-24T22:54:11.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4035030 con 0x7f240c108680 2024-04-24T22:54:11.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.192+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c1aaf70 con 0x7f240c108680 2024-04-24T22:54:11.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.193+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c109330 con 0x7f240c108680 2024-04-24T22:54:11.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.195+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f23f4042070 con 0x7f240c108680 2024-04-24T22:54:11.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.196+0000 7f2409600640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 0x7f23f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.197+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f23f40efff0 con 0x7f240c108680 2024-04-24T22:54:11.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.197+0000 7f240be00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 0x7f23f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.197+0000 7f240be00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 0x7f23f00a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0095f0 tx=0x7f23fc005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:11.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.199+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f23f40bc580 con 0x7f240c108680 2024-04-24T22:54:11.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.440+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f240c1100f0 con 0x7f240c108680 2024-04-24T22:54:11.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.465+0000 7f2409600640 1 -- 172.21.15.111:0/1830554681 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f23f40c0880 con 0x7f240c108680 2024-04-24T22:54:11.467 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T22:54:11.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.468+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 msgr2=0x7f23f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.468+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 0x7f23f00a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0095f0 tx=0x7f23fc005f90 comp rx=0 tx=0).stop 2024-04-24T22:54:11.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.468+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 msgr2=0x7f240c19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.468+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c19e990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23f402a9a0 tx=0x7f23f4003b90 comp rx=0 tx=0).stop 2024-04-24T22:54:11.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 shutdown_connections 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c10dd00 0x7f240c1a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f23f00a2d00 0x7f23f00a51c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c108680 0x7f240c19e990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 --2- 172.21.15.111:0/1830554681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c107ce0 0x7f240c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 >> 172.21.15.111:0/1830554681 conn(0x7f240c077560 msgr2=0x7f240c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 shutdown_connections 2024-04-24T22:54:11.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.469+0000 7f2411a00640 1 -- 172.21.15.111:0/1830554681 wait complete. 2024-04-24T22:54:11.489 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 -- 172.21.15.111:0/301190017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 msgr2=0x7fcff8108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 --2- 172.21.15.111:0/301190017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff8108b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009900 tx=0x7fcfe802a3f0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 -- 172.21.15.111:0/301190017 shutdown_connections 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 --2- 172.21.15.111:0/301190017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8114890 0x7fcff8116c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 --2- 172.21.15.111:0/301190017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8109060 0x7fcff8114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 --2- 172.21.15.111:0/301190017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff8108b20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 -- 172.21.15.111:0/301190017 >> 172.21.15.111:0/301190017 conn(0x7fcff806ccb0 msgr2=0x7fcff806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:11.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 -- 172.21.15.111:0/301190017 shutdown_connections 2024-04-24T22:54:11.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.671+0000 7fcffdc00640 1 -- 172.21.15.111:0/301190017 wait complete. 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.672+0000 7fcffdc00640 1 Processor -- start 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.672+0000 7fcffdc00640 1 -- start start 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff81a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 0x7fcff81a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcff811a310 con 0x7fcff8109060 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcff811a190 con 0x7fcff8108740 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcffdc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcff811a490 con 0x7fcff8114890 2024-04-24T22:54:11.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41738/0 (socket says 172.21.15.111:41738) 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcfefe00640 1 -- 172.21.15.111:0/603785036 learned_addr learned my addr 172.21.15.111:0/603785036 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcff6a00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff81a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.673+0000 7fcff7400640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 0x7fcff81a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 -- 172.21.15.111:0/603785036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 msgr2=0x7fcff81a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 0x7fcff81a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 -- 172.21.15.111:0/603785036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 msgr2=0x7fcff81a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff81a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 -- 172.21.15.111:0/603785036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81a3110 con 0x7fcff8109060 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcff7400640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 0x7fcff81a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfefe00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e9f0 tx=0x7fcfe000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cd80 con 0x7fcff8109060 2024-04-24T22:54:11.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe0005500 con 0x7fcff8109060 2024-04-24T22:54:11.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a3400 con 0x7fcff8109060 2024-04-24T22:54:11.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0010690 con 0x7fcff8109060 2024-04-24T22:54:11.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.674+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81af6a0 con 0x7fcff8109060 2024-04-24T22:54:11.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.675+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff8068560 con 0x7fcff8109060 2024-04-24T22:54:11.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.677+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcfe00046c0 con 0x7fcff8109060 2024-04-24T22:54:11.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.678+0000 7fcfef400640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 0x7fcfdc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:11.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.679+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8771+0+0 (secure 0 0 0) 0x7fcfe00cfa80 con 0x7fcff8109060 2024-04-24T22:54:11.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.679+0000 7fcff6a00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 0x7fcfdc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:11.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.679+0000 7fcff6a00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 0x7fcfdc0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8006010 tx=0x7fcfe8000bc0 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:11.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.683+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcfe00d1e20 con 0x7fcff8109060 2024-04-24T22:54:11.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2196216093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/770865654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.750 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: ignoring --setuser ceph since I am not root 2024-04-24T22:54:11.751 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: ignoring --setgroup ceph since I am not root 2024-04-24T22:54:11.751 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mds[63731]: -- 172.21.15.186:0/2681795122 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f08ee84e0 con 0x560f08ebec00 2024-04-24T22:54:11.751 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mds[63731]: -- 172.21.15.186:0/2681795122 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f08eeba40 con 0x560f08ebec00 2024-04-24T22:54:11.751 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: starting mds.c at 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: ignoring --setuser ceph since I am not root 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: ignoring --setgroup ceph since I am not root 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mds[63411]: -- 172.21.15.186:0/3964405355 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f4b673a40 con 0x564f4b646800 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: starting mds.f at 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: ignoring --setuser ceph since I am not root 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: ignoring --setgroup ceph since I am not root 2024-04-24T22:54:11.752 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-mds[64039]: -- 172.21.15.186:0/2529953310 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b2e969a40 con 0x555b2e93c800 2024-04-24T22:54:11.753 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:54:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: starting mds.i at 2024-04-24T22:54:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2196216093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:11 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/770865654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a[27742]: 2024-04-24T22:54:11.454+0000 7f1d34c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:54:11.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2196216093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/770865654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: ignoring --setuser ceph since I am not root 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: ignoring --setgroup ceph since I am not root 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-mds[62167]: -- 172.21.15.171:0/3277584367 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa86f99a40 con 0x55aa86f6d000 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: starting mds.b at 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: ignoring --setuser ceph since I am not root 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: ignoring --setgroup ceph since I am not root 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-mds[61856]: -- 172.21.15.171:0/3735248137 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b895fada40 con 0x55b895f80c00 2024-04-24T22:54:11.907 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:54:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: starting mds.h at 2024-04-24T22:54:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:11.906+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcff81af8d0 con 0x7fcff8109060 2024-04-24T22:54:12.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.459+0000 7fcfef400640 1 -- 172.21.15.111:0/603785036 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fcfe009c540 con 0x7fcff8109060 2024-04-24T22:54:12.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.465+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 msgr2=0x7fcfdc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:12.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.465+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 0x7fcfdc0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8006010 tx=0x7fcfe8000bc0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.465+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 msgr2=0x7fcff81a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:12.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.465+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e9f0 tx=0x7fcfe000edf0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 shutdown_connections 2024-04-24T22:54:12.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fcfdc0a2d00 0x7fcfdc0a51c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff8114890 0x7fcff81a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8109060 0x7fcff81a8d20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 --2- 172.21.15.111:0/603785036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcff8108740 0x7fcff81a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 >> 172.21.15.111:0/603785036 conn(0x7fcff806ccb0 msgr2=0x7fcff810a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:12.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.467+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 shutdown_connections 2024-04-24T22:54:12.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.468+0000 7fcffdc00640 1 -- 172.21.15.111:0/603785036 wait complete. 2024-04-24T22:54:12.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T22:54:12.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 -- 172.21.15.111:0/760423148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 msgr2=0x7f8a1c1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:12.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 --2- 172.21.15.111:0/760423148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c1103f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009ca0 tx=0x7f8a1402a950 comp rx=0 tx=0).stop 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 -- 172.21.15.111:0/760423148 shutdown_connections 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 --2- 172.21.15.111:0/760423148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c1103f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 --2- 172.21.15.111:0/760423148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 0x7f8a1c10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 --2- 172.21.15.111:0/760423148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 -- 172.21.15.111:0/760423148 >> 172.21.15.111:0/760423148 conn(0x7f8a1c0fd250 msgr2=0x7f8a1c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 -- 172.21.15.111:0/760423148 shutdown_connections 2024-04-24T22:54:12.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.689+0000 7f8a20c00640 1 -- 172.21.15.111:0/760423148 wait complete. 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 Processor -- start 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 -- start start 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 0x7f8a1c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c113d40 con 0x7f8a1c10dfa0 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c113bc0 con 0x7f8a1c074490 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.690+0000 7f8a20c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c113ec0 con 0x7f8a1c073b70 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a1a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:58046/0 (socket says 172.21.15.111:58046) 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 -- 172.21.15.111:0/2085367318 learned_addr learned my addr 172.21.15.111:0/2085367318 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19400640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 0x7f8a1c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:12.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 -- 172.21.15.111:0/2085367318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 msgr2=0x7f8a1c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:12.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 0x7f8a1c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 -- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 msgr2=0x7f8a1c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:12.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:12.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 -- 172.21.15.111:0/2085367318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c19f220 con 0x7f8a1c073b70 2024-04-24T22:54:12.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.691+0000 7f8a19e00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e9f0 tx=0x7f8a0c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.692+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00cd80 con 0x7f8a1c073b70 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.692+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c19f510 con 0x7f8a1c073b70 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.692+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c1a6da0 con 0x7f8a1c073b70 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.692+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0c005500 con 0x7f8a1c073b70 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.692+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c010690 con 0x7f8a1c073b70 2024-04-24T22:54:12.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.693+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1c0689b0 con 0x7f8a1c073b70 2024-04-24T22:54:12.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.695+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8a0c0046c0 con 0x7f8a1c073b70 2024-04-24T22:54:12.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.697+0000 7f8a12a00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 0x7f8a000a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:12.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.697+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8771+0+0 (secure 0 0 0) 0x7f8a0c0cfae0 con 0x7f8a1c073b70 2024-04-24T22:54:12.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.697+0000 7f8a19400640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 0x7f8a000a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:12.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.698+0000 7f8a19400640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 0x7f8a000a5240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ff8e0 tx=0x7f8a0800b410 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:12.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.699+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a0c053740 con 0x7f8a1c073b70 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: pgmap v382: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1830554681' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T22:54:12.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T22:54:12.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T22:54:12.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:12 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: pgmap v382: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1830554681' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T22:54:12.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:12 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:54:12.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:12.885+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a1c109120 con 0x7f8a1c073b70 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: pgmap v382: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1830554681' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T22:54:12.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T22:54:12.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:12 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.468+0000 7f8a12a00640 1 -- 172.21.15.111:0/2085367318 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f8a0c09c5a0 con 0x7f8a1c073b70 2024-04-24T22:54:13.470 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2024-04-24T22:54:13.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.473+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 msgr2=0x7f8a000a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.473+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 0x7f8a000a5240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ff8e0 tx=0x7f8a0800b410 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.473+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 msgr2=0x7f8a1c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.473+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e9f0 tx=0x7f8a0c00edf0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 shutdown_connections 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a1c10dfa0 0x7f8a1c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a1c074490 0x7f8a1c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7f8a000a2d80 0x7f8a000a5240 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 --2- 172.21.15.111:0/2085367318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1c073b70 0x7f8a1c19a280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 >> 172.21.15.111:0/2085367318 conn(0x7f8a1c0fd250 msgr2=0x7f8a1c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:13.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.474+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 shutdown_connections 2024-04-24T22:54:13.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.475+0000 7f8a20c00640 1 -- 172.21.15.111:0/2085367318 wait complete. 2024-04-24T22:54:13.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 -- 172.21.15.111:0/2212139542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 msgr2=0x7fadb4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 --2- 172.21.15.111:0/2212139542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb4107b00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fada4009930 tx=0x7fada402a420 comp rx=0 tx=0).stop 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 -- 172.21.15.111:0/2212139542 shutdown_connections 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 --2- 172.21.15.111:0/2212139542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 --2- 172.21.15.111:0/2212139542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 --2- 172.21.15.111:0/2212139542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb4107b00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 -- 172.21.15.111:0/2212139542 >> 172.21.15.111:0/2212139542 conn(0x7fadb4078b90 msgr2=0x7fadb4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 -- 172.21.15.111:0/2212139542 shutdown_connections 2024-04-24T22:54:13.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.698+0000 7fadb9000640 1 -- 172.21.15.111:0/2212139542 wait complete. 2024-04-24T22:54:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.699+0000 7fadb9000640 1 Processor -- start 2024-04-24T22:54:13.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.699+0000 7fadb9000640 1 -- start start 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.699+0000 7fadb9000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb9000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb9000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4111b80 con 0x7fadb4107720 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb9000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4111a00 con 0x7fadb4101710 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb9000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4111d00 con 0x7fadb41020d0 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41768/0 (socket says 172.21.15.111:41768) 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb2800640 1 -- 172.21.15.111:0/1803137452 learned_addr learned my addr 172.21.15.111:0/1803137452 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.111:58058/0 (socket says 172.21.15.111:58058) 2024-04-24T22:54:13.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 -- 172.21.15.111:0/1803137452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 msgr2=0x7fadb41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 -- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 msgr2=0x7fadb419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb1e00640 1 -- 172.21.15.111:0/1803137452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb419d050 con 0x7fadb4101710 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.700+0000 7fadb2800640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadb1e00640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4198060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fada402a940 tx=0x7fada4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada403f070 con 0x7fadb4101710 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb419d2e0 con 0x7fadb4101710 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada4031c90 con 0x7fadb4101710 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4035030 con 0x7fadb4101710 2024-04-24T22:54:13.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.701+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb41a4bd0 con 0x7fadb4101710 2024-04-24T22:54:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.702+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb40689b0 con 0x7fadb4101710 2024-04-24T22:54:13.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.704+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fada4042070 con 0x7fadb4101710 2024-04-24T22:54:13.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.705+0000 7fadaaa00640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 0x7fad980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:13.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.706+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8246+0+0 (secure 0 0 0) 0x7fada40f01e0 con 0x7fadb4101710 2024-04-24T22:54:13.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.706+0000 7fadb1400640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 0x7fad980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:13.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.706+0000 7fadb1400640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 0x7fad980a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadb4101000 tx=0x7fada0005f90 comp rx=0 tx=0).ready entity=mgr.25204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:13.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.708+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fada40bc860 con 0x7fadb4101710 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: Cluster is now healthy 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:54:13.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6834/1447672017,v1:172.21.15.186:6837/1447672017] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6835/1425553810,v1:172.21.15.186:6838/1425553810] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6836/819529302,v1:172.21.15.171:6838/819529302] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.186:6836/2812503749,v1:172.21.15.186:6839/2812503749] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.171:6837/1581200558,v1:172.21.15.171:6839/1581200558] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: mds.? [v2:172.21.15.111:6838/4002944745,v1:172.21.15.111:6839/4002944745] up:boot 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: fsmap 12 up:standby 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/2085367318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:13 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: Cluster is now healthy 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:54:13.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6834/1447672017,v1:172.21.15.186:6837/1447672017] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6835/1425553810,v1:172.21.15.186:6838/1425553810] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6836/819529302,v1:172.21.15.171:6838/819529302] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.186:6836/2812503749,v1:172.21.15.186:6839/2812503749] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.171:6837/1581200558,v1:172.21.15.171:6839/1581200558] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: mds.? [v2:172.21.15.111:6838/4002944745,v1:172.21.15.111:6839/4002944745] up:boot 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: fsmap 12 up:standby 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/2085367318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.785 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:13 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:54:13.906 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: Cluster is now healthy 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/603785036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='mgr.25204 172.21.15.111:0/1698545485' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6834/1447672017,v1:172.21.15.186:6837/1447672017] up:boot 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6835/1425553810,v1:172.21.15.186:6838/1425553810] up:boot 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6836/819529302,v1:172.21.15.171:6838/819529302] up:boot 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.186:6836/2812503749,v1:172.21.15.186:6839/2812503749] up:boot 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.171:6837/1581200558,v1:172.21.15.171:6839/1581200558] up:boot 2024-04-24T22:54:13.907 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: mds.? [v2:172.21.15.111:6838/4002944745,v1:172.21.15.111:6839/4002944745] up:boot 2024-04-24T22:54:13.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: fsmap 12 up:standby 2024-04-24T22:54:13.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/2085367318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.908 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:13 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:13.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:13.909+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fadb4109c80 con 0x7fadb4101710 2024-04-24T22:54:14.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.481+0000 7fadaaa00640 1 -- 172.21.15.111:0/1803137452 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7fada40c0b60 con 0x7fadb4101710 2024-04-24T22:54:14.483 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2024-04-24T22:54:14.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 msgr2=0x7fad980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:14.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 0x7fad980a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadb4101000 tx=0x7fada0005f90 comp rx=0 tx=0).stop 2024-04-24T22:54:14.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 msgr2=0x7fadb4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:14.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4198060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fada402a940 tx=0x7fada4003b20 comp rx=0 tx=0).stop 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 shutdown_connections 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb4107720 0x7fadb419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.111:6800/1932258627,v1:172.21.15.111:6801/1932258627] conn(0x7fad980a2d00 0x7fad980a51c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb41020d0 0x7fadb41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 --2- 172.21.15.111:0/1803137452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadb4101710 0x7fadb4198060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 >> 172.21.15.111:0/1803137452 conn(0x7fadb4078b90 msgr2=0x7fadb40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.487+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 shutdown_connections 2024-04-24T22:54:14.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-24T22:54:14.488+0000 7fadb9000640 1 -- 172.21.15.111:0/1803137452 wait complete. 2024-04-24T22:54:14.514 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:54:14.586 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:54:14.620 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:54:14.654 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T22:54:14.655 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T22:54:14.710 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T22:54:14.728 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T22:54:14.734 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:14 smithi171 ceph-mon[32337]: pgmap v385: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T22:54:14.734 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:14 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T22:54:14.734 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:14 smithi171 ceph-mon[32337]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T22:54:14.734 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:14 smithi171 ceph-mon[32337]: from='client.? 172.21.15.111:0/1803137452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:14.734 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:14 smithi171 ceph-mon[32337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:14.745 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T22:54:14.745 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T22:54:14.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a 2024-04-24T22:54:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:14 smithi186 ceph-mon[32753]: pgmap v385: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T22:54:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:14 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T22:54:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:14 smithi186 ceph-mon[32753]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T22:54:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:14 smithi186 ceph-mon[32753]: from='client.? 172.21.15.111:0/1803137452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:14.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:14 smithi186 ceph-mon[32753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 ceph-mon[27746]: pgmap v385: 97 pgs: 97 active+clean; 3.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T22:54:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T22:54:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 ceph-mon[27746]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T22:54:14.783 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 ceph-mon[27746]: from='client.? 172.21.15.111:0/1803137452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:14.784 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 ceph-mon[27746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:54:15.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:14 smithi111 systemd[1]: Stopping Ceph mon.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:15.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a[27742]: 2024-04-24T22:54:15.276+0000 7f1d3ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:15.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a[27742]: 2024-04-24T22:54:15.276+0000 7f1d3ba00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T22:54:15.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 podman[88517]: 2024-04-24 22:54:15.318451885 +0000 UTC m=+0.177820142 container died e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T22:54:15.999 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 podman[88517]: 2024-04-24 22:54:15.746595296 +0000 UTC m=+0.605963620 container cleanup e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T22:54:15.999 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 bash[88517]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a 2024-04-24T22:54:16.283 INFO:journalctl@ceph.mon.a.smithi111.stdout:Apr 24 22:54:15 smithi111 podman[88531]: 2024-04-24 22:54:15.997393837 +0000 UTC m=+0.665185031 container remove e592b297f0c50d41c8d8390f65600533f51fa06ca67a9fb7362fce44318583e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T22:54:16.532 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T22:54:16.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:16.706 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T22:54:16.706 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T22:54:16.706 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.b 2024-04-24T22:54:17.008 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:16 smithi171 systemd[1]: Stopping Ceph mon.b for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:17.263 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-b[32333]: 2024-04-24T22:54:17.007+0000 7f5a16b79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:17.263 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-b[32333]: 2024-04-24T22:54:17.007+0000 7f5a16b79700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T22:54:17.534 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:17 smithi171 podman[68308]: 2024-04-24 22:54:17.262849862 +0000 UTC m=+0.363697521 container died 2dee2d89540c280f3c2607a95651c6f962e44664a7f2a25d8b42e48bc49cf151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T22:54:17.902 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:17 smithi171 podman[68308]: 2024-04-24 22:54:17.642055363 +0000 UTC m=+0.742903022 container cleanup 2dee2d89540c280f3c2607a95651c6f962e44664a7f2a25d8b42e48bc49cf151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T22:54:17.902 INFO:journalctl@ceph.mon.b.smithi171.stdout:Apr 24 22:54:17 smithi171 bash[68308]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-b 2024-04-24T22:54:18.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T22:54:18.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:18.086 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T22:54:18.086 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T22:54:18.086 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.c 2024-04-24T22:54:18.386 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:18 smithi186 systemd[1]: Stopping Ceph mon.c for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:18.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-c[32741]: 2024-04-24T22:54:18.384+0000 7f85ee4b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:18.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-c[32741]: 2024-04-24T22:54:18.384+0000 7f85ee4b6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T22:54:18.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:18 smithi186 podman[69597]: 2024-04-24 22:54:18.640581464 +0000 UTC m=+0.358597107 container died d82c1bb37bb8c9ecbf5148c5ac8757815dd2d8d1662315b17dab3db9f1f4cf23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:54:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:19 smithi186 podman[69597]: 2024-04-24 22:54:19.026485947 +0000 UTC m=+0.744501633 container cleanup d82c1bb37bb8c9ecbf5148c5ac8757815dd2d8d1662315b17dab3db9f1f4cf23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:19.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 24 22:54:19 smithi186 bash[69597]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mon-c 2024-04-24T22:54:19.463 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T22:54:19.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:19.506 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T22:54:19.506 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T22:54:19.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x 2024-04-24T22:54:20.033 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:19 smithi111 systemd[1]: Stopping Ceph mgr.x for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:20.511 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:54:20.064+0000 7fcca6c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:20.511 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x[27988]: 2024-04-24T22:54:20.064+0000 7fcca6c00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T22:54:20.511 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:20 smithi111 podman[88639]: 2024-04-24 22:54:20.159807126 +0000 UTC m=+0.227952144 container died a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T22:54:21.284 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:20 smithi111 podman[88639]: 2024-04-24 22:54:20.860096733 +0000 UTC m=+0.928241788 container cleanup a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T22:54:21.284 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:20 smithi111 bash[88639]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x 2024-04-24T22:54:21.284 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Apr 24 22:54:21 smithi111 podman[88653]: 2024-04-24 22:54:21.077844444 +0000 UTC m=+0.901548571 container remove a50fb1ee3480e6e8e807cbd9df8b90047f6ab4b8f1d4c283da2efeb03d080a9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T22:54:21.594 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T22:54:21.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:21.703 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T22:54:21.703 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T22:54:21.703 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.y 2024-04-24T22:54:22.061 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:54:21 smithi171 systemd[1]: Stopping Ceph mgr.y for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:22.330 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:54:22 smithi171 podman[68429]: 2024-04-24 22:54:22.060315388 +0000 UTC m=+0.167908957 container died 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True) 2024-04-24T22:54:22.657 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:54:22 smithi171 podman[68429]: 2024-04-24 22:54:22.429092486 +0000 UTC m=+0.536686055 container cleanup 8df2310a3196e67bddd8e212351fab870d65f0d2ef411df7a39531ab12c70cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T22:54:22.657 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Apr 24 22:54:22 smithi171 bash[68429]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-y 2024-04-24T22:54:22.804 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T22:54:22.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:22.839 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T22:54:22.839 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T22:54:22.839 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.z 2024-04-24T22:54:23.169 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:54:22 smithi186 systemd[1]: Stopping Ceph mgr.z for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:23.474 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:54:23 smithi186 podman[69719]: 2024-04-24 22:54:23.168506469 +0000 UTC m=+0.147377750 container stop 203973cd9bea991642c81d2b0838f360f0afcab80f1fdf725702002ff32f2634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T22:54:23.474 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:54:23 smithi186 podman[69719]: 2024-04-24 22:54:23.194806072 +0000 UTC m=+0.173677354 container died 203973cd9bea991642c81d2b0838f360f0afcab80f1fdf725702002ff32f2634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T22:54:23.724 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:54:23 smithi186 podman[69719]: 2024-04-24 22:54:23.566549114 +0000 UTC m=+0.545420394 container cleanup 203973cd9bea991642c81d2b0838f360f0afcab80f1fdf725702002ff32f2634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T22:54:23.724 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 24 22:54:23 smithi186 bash[69719]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mgr-z 2024-04-24T22:54:23.934 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T22:54:23.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:23.966 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T22:54:23.967 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T22:54:23.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.0 2024-04-24T22:54:24.533 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:24 smithi111 systemd[1]: Stopping Ceph osd.0 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:25.033 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0[42944]: 2024-04-24T22:54:24.561+0000 7f2937800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:25.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0[42944]: 2024-04-24T22:54:24.561+0000 7f2937800700 -1 osd.0 104 *** Got signal Terminated *** 2024-04-24T22:54:25.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0[42944]: 2024-04-24T22:54:24.561+0000 7f2937800700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:30.223 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:29 smithi111 podman[88759]: 2024-04-24 22:54:29.913663285 +0000 UTC m=+5.490496678 container died 6097849997f3fc1264ce24e52726e6bd42b4187aaa82d915a84894609aab9223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:30.533 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:30 smithi111 podman[88759]: 2024-04-24 22:54:30.405836617 +0000 UTC m=+5.982670057 container cleanup 6097849997f3fc1264ce24e52726e6bd42b4187aaa82d915a84894609aab9223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T22:54:30.533 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:30 smithi111 bash[88759]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0 2024-04-24T22:54:31.033 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:30 smithi111 podman[88776]: 2024-04-24 22:54:30.723812391 +0000 UTC m=+0.801537467 container remove 6097849997f3fc1264ce24e52726e6bd42b4187aaa82d915a84894609aab9223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T22:54:31.624 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:31 smithi111 podman[88864]: 2024-04-24 22:54:31.276424574 +0000 UTC m=+0.038667503 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:31.624 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:31 smithi111 podman[88864]: 2024-04-24 22:54:31.434056497 +0000 UTC m=+0.196299445 container create cc991719082169dc909507672b8687255e5429f713f2fe552bf17503ebf7f7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T22:54:32.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:31 smithi111 podman[88864]: 2024-04-24 22:54:31.745822878 +0000 UTC m=+0.508065833 container init cc991719082169dc909507672b8687255e5429f713f2fe552bf17503ebf7f7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T22:54:32.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:31 smithi111 podman[88864]: 2024-04-24 22:54:31.756712144 +0000 UTC m=+0.518955090 container start cc991719082169dc909507672b8687255e5429f713f2fe552bf17503ebf7f7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T22:54:32.034 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:31 smithi111 podman[88864]: 2024-04-24 22:54:31.827839497 +0000 UTC m=+0.590082471 container attach cc991719082169dc909507672b8687255e5429f713f2fe552bf17503ebf7f7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T22:54:32.533 INFO:journalctl@ceph.osd.0.smithi111.stdout:Apr 24 22:54:32 smithi111 podman[88864]: 2024-04-24 22:54:32.21232337 +0000 UTC m=+0.974566348 container died cc991719082169dc909507672b8687255e5429f713f2fe552bf17503ebf7f7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T22:54:32.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T22:54:33.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:33.033 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T22:54:33.033 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T22:54:33.033 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.1 2024-04-24T22:54:33.401 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:33 smithi171 systemd[1]: Stopping Ceph osd.1 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:33.656 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1[36971]: 2024-04-24T22:54:33.399+0000 7f1365906700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:33.656 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1[36971]: 2024-04-24T22:54:33.399+0000 7f1365906700 -1 osd.1 104 *** Got signal Terminated *** 2024-04-24T22:54:33.656 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1[36971]: 2024-04-24T22:54:33.399+0000 7f1365906700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:39.125 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:38 smithi171 podman[68553]: 2024-04-24 22:54:38.798038542 +0000 UTC m=+5.577788287 container died 7845ac9a9838883e4f1c5cb5eafabe821906bca62f2a3ed8bd888d092142a28b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T22:54:39.404 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68553]: 2024-04-24 22:54:39.210875821 +0000 UTC m=+5.990625553 container cleanup 7845ac9a9838883e4f1c5cb5eafabe821906bca62f2a3ed8bd888d092142a28b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:39.404 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 bash[68553]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1 2024-04-24T22:54:39.659 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68583]: 2024-04-24 22:54:39.403732811 +0000 UTC m=+0.601457542 container remove 7845ac9a9838883e4f1c5cb5eafabe821906bca62f2a3ed8bd888d092142a28b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:54:40.041 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68678]: 2024-04-24 22:54:39.627271257 +0000 UTC m=+0.018262643 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:40.041 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68678]: 2024-04-24 22:54:39.738049965 +0000 UTC m=+0.129041336 container create 7862d9f84e5db8e3c39c1890af0108e10e28a81828c8b8caf7b64c00f8aec679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:40.041 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68678]: 2024-04-24 22:54:39.948758155 +0000 UTC m=+0.339749527 container init 7862d9f84e5db8e3c39c1890af0108e10e28a81828c8b8caf7b64c00f8aec679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T22:54:40.041 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:39 smithi171 podman[68678]: 2024-04-24 22:54:39.951823715 +0000 UTC m=+0.342815087 container start 7862d9f84e5db8e3c39c1890af0108e10e28a81828c8b8caf7b64c00f8aec679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:54:40.041 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:40 smithi171 podman[68678]: 2024-04-24 22:54:40.040488108 +0000 UTC m=+0.431479502 container attach 7862d9f84e5db8e3c39c1890af0108e10e28a81828c8b8caf7b64c00f8aec679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T22:54:40.406 INFO:journalctl@ceph.osd.1.smithi171.stdout:Apr 24 22:54:40 smithi171 podman[68678]: 2024-04-24 22:54:40.24153343 +0000 UTC m=+0.632524802 container died 7862d9f84e5db8e3c39c1890af0108e10e28a81828c8b8caf7b64c00f8aec679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T22:54:40.843 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T22:54:40.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:40.885 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T22:54:40.885 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T22:54:40.885 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.2 2024-04-24T22:54:41.177 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:40 smithi186 systemd[1]: Stopping Ceph osd.2 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:41.441 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2[37937]: 2024-04-24T22:54:41.175+0000 7f8ff30c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:41.441 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2[37937]: 2024-04-24T22:54:41.175+0000 7f8ff30c4700 -1 osd.2 104 *** Got signal Terminated *** 2024-04-24T22:54:41.441 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2[37937]: 2024-04-24T22:54:41.175+0000 7f8ff30c4700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:46.952 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:46 smithi186 podman[69846]: 2024-04-24 22:54:46.668516637 +0000 UTC m=+5.593889508 container died 687b851ce8ef245be0e966baaa8e95df3bdc5ca4ddd80dc9747695e62620b8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T22:54:47.249 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69846]: 2024-04-24 22:54:47.050537214 +0000 UTC m=+5.975910095 container cleanup 687b851ce8ef245be0e966baaa8e95df3bdc5ca4ddd80dc9747695e62620b8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T22:54:47.249 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 bash[69846]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2 2024-04-24T22:54:47.712 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69875]: 2024-04-24 22:54:47.360753641 +0000 UTC m=+0.690503391 container remove 687b851ce8ef245be0e966baaa8e95df3bdc5ca4ddd80dc9747695e62620b8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T22:54:47.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69965]: 2024-04-24 22:54:47.614445926 +0000 UTC m=+0.017987794 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:47.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69965]: 2024-04-24 22:54:47.71993486 +0000 UTC m=+0.123476714 container create 0d607782cdbb945681ec9f9181225ae849b43cdaeecec20feef07ceb1c63ed27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T22:54:47.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69965]: 2024-04-24 22:54:47.93906359 +0000 UTC m=+0.342605448 container init 0d607782cdbb945681ec9f9181225ae849b43cdaeecec20feef07ceb1c63ed27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T22:54:47.999 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:47 smithi186 podman[69965]: 2024-04-24 22:54:47.94236779 +0000 UTC m=+0.345909648 container start 0d607782cdbb945681ec9f9181225ae849b43cdaeecec20feef07ceb1c63ed27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T22:54:48.251 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:48 smithi186 podman[69965]: 2024-04-24 22:54:48.072347202 +0000 UTC m=+0.475889059 container attach 0d607782cdbb945681ec9f9181225ae849b43cdaeecec20feef07ceb1c63ed27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T22:54:48.252 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 24 22:54:48 smithi186 podman[69965]: 2024-04-24 22:54:48.235470823 +0000 UTC m=+0.639012668 container died 0d607782cdbb945681ec9f9181225ae849b43cdaeecec20feef07ceb1c63ed27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:54:48.878 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T22:54:48.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:48.917 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T22:54:48.918 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T22:54:48.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.3 2024-04-24T22:54:49.286 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:49 smithi111 systemd[1]: Stopping Ceph osd.3 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:49.784 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:49.523+0000 7f096f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:49.784 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:49.523+0000 7f096f800700 -1 osd.3 104 *** Got signal Terminated *** 2024-04-24T22:54:49.784 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:49 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:49.523+0000 7f096f800700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:50.533 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:50 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:50.100+0000 7f096ba00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:28.385246+0000 front 2024-04-24T22:54:28.385856+0000 (oldest deadline 2024-04-24T22:54:50.085563+0000) 2024-04-24T22:54:50.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:50.267+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:51.533 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:51.077+0000 7f096ba00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:28.385246+0000 front 2024-04-24T22:54:28.385856+0000 (oldest deadline 2024-04-24T22:54:50.085563+0000) 2024-04-24T22:54:51.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:51.275+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:52.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:51 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:51.576+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:52.533 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:52 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:52.073+0000 7f096ba00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:28.385246+0000 front 2024-04-24T22:54:28.385856+0000 (oldest deadline 2024-04-24T22:54:50.085563+0000) 2024-04-24T22:54:52.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:52.316+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:53.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:52 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:52.539+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:53.361 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:53.049+0000 7f096ba00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:28.385246+0000 front 2024-04-24T22:54:28.385856+0000 (oldest deadline 2024-04-24T22:54:50.085563+0000) 2024-04-24T22:54:53.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:53.350+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:53.322+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:53.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:53.567+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:53.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:53 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:53.359+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:54.283 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3[49757]: 2024-04-24T22:54:54.017+0000 7f096ba00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:28.385246+0000 front 2024-04-24T22:54:28.385856+0000 (oldest deadline 2024-04-24T22:54:50.085563+0000) 2024-04-24T22:54:54.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:54 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:54.388+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:54.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:54.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:54.335+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:54.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:54 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:54.537+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:55.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:54.839+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:54:55.265 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:54 smithi111 podman[88981]: 2024-04-24 22:54:54.999273369 +0000 UTC m=+5.609843908 container died e5f98619ce3480a20a630b2bd0a9d5b51047c711520152f52c45effa0085c388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:54:55.533 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:55 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:55.356+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:55.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:55 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:55.509+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:55.534 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:55 smithi111 podman[88981]: 2024-04-24 22:54:55.416632099 +0000 UTC m=+6.027202654 container cleanup e5f98619ce3480a20a630b2bd0a9d5b51047c711520152f52c45effa0085c388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T22:54:55.534 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:55 smithi111 bash[88981]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3 2024-04-24T22:54:55.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:55 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:55.398+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:55.340+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:56.033 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:55 smithi111 podman[89000]: 2024-04-24 22:54:55.659826972 +0000 UTC m=+0.643666866 container remove e5f98619ce3480a20a630b2bd0a9d5b51047c711520152f52c45effa0085c388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T22:54:56.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:55.841+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:54:56.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:56 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:56.390+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:56.745 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:56.348+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:56.745 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:56 smithi111 podman[89087]: 2024-04-24 22:54:56.426886743 +0000 UTC m=+0.032160209 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:56.746 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:56 smithi111 podman[89087]: 2024-04-24 22:54:56.577822719 +0000 UTC m=+0.183096219 container create 90dc8f8458eeff279db6a9e5f4f44637ddd3e7b13b1fb43d34af67388d2b7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T22:54:56.746 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:56 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:56.486+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:56.301+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:57.034 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:56 smithi111 podman[89087]: 2024-04-24 22:54:56.87474075 +0000 UTC m=+0.480014295 container init 90dc8f8458eeff279db6a9e5f4f44637ddd3e7b13b1fb43d34af67388d2b7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T22:54:57.034 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:56 smithi111 podman[89087]: 2024-04-24 22:54:56.886858151 +0000 UTC m=+0.492131683 container start 90dc8f8458eeff279db6a9e5f4f44637ddd3e7b13b1fb43d34af67388d2b7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T22:54:57.034 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:56 smithi111 podman[89087]: 2024-04-24 22:54:56.972014644 +0000 UTC m=+0.577288163 container attach 90dc8f8458eeff279db6a9e5f4f44637ddd3e7b13b1fb43d34af67388d2b7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T22:54:57.252 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:56.841+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:54:57.682 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:57.320+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:57.682 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:57 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:57.519+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:57.682 INFO:journalctl@ceph.osd.3.smithi111.stdout:Apr 24 22:54:57 smithi111 podman[89087]: 2024-04-24 22:54:57.336268129 +0000 UTC m=+0.941541651 container died 90dc8f8458eeff279db6a9e5f4f44637ddd3e7b13b1fb43d34af67388d2b7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T22:54:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:57.349+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:57.906 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:57 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:57.416+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:58.011 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T22:54:58.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:54:58.101 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T22:54:58.101 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T22:54:58.101 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.4 2024-04-24T22:54:58.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:57.867+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:54:58.405 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:54:58 smithi171 systemd[1]: Stopping Ceph osd.4 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:58.405 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:58.371+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:58.656 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:54:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:54:58.403+0000 7f286404d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:58.656 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:54:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:54:58.403+0000 7f286404d700 -1 osd.4 104 *** Got signal Terminated *** 2024-04-24T22:54:58.656 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:54:58 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:54:58.403+0000 7f286404d700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:58.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:58.316+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:58.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:58 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:58.333+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:58.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:58 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:58.537+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:54:59.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:58.843+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:54:59.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:54:59 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:54:59.361+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:54:59.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:59.300+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:54:59.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:54:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:54:59.300+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:54:59.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:59.343+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:54:59.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:54:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:54:59.343+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:54:59.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:54:59 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:54:59.533+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:00.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:54:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:54:59.819+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:00.348 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:00 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:00.091+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:00.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:00 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:00.347+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:00.308+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:00.308+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:00.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:00.515+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:00.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:00.347+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:00.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:00 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:00.347+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:01.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:00.850+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:01.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:01 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:01.120+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:01.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:01 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:01.365+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:01.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:01 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:01.313+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:01.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:01 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:01.313+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:01.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:01 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:01.564+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:01.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:01 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:01.364+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:01.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:01 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:01.364+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:02.072 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:01 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:01.808+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:02.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:02 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:02.093+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:02.499 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:02 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:02.070+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:02.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:02 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:02.287+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:02.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:02 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:02.287+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:02.600 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:02 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:02.334+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:02.601 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:02 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:02.334+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:02.906 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:02 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4[43517]: 2024-04-24T22:55:02.507+0000 7f285ce3a700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.969300+0000 front 2024-04-24T22:54:37.969338+0000 (oldest deadline 2024-04-24T22:55:02.069486+0000) 2024-04-24T22:55:02.906 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:02 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:02.411+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:02.907 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:02 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:02.411+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:03.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:02 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:02.599+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:03.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:02 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:02.599+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:03.046 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:02 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:02.780+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:03.046 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:02 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:02.780+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:03.370 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:03 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:03.106+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:03.499 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:03 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:03.044+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:03.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:03 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:03.294+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:03.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:03 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:03.294+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:03.590 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:03 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:03.320+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:03.590 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:03 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:03.320+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:03.646 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:03 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:03.368+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:03.646 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:03 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:03.368+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:03.906 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:03 smithi171 podman[68784]: 2024-04-24 22:55:03.852855417 +0000 UTC m=+5.559959984 container died bd3bce09c31c23f978bf3deeeff18c78af61b8983760e717d57fc34857a43d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T22:55:04.021 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:03 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:03.762+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:04.021 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:03 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:03.762+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:04.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:03 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:03.588+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:04.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:03 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:03.588+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:04.304 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:04 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:04.020+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:04.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:04 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:04.079+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:04.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:04 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:04.319+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:04.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:04 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:04.319+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:04.407 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:04 smithi171 podman[68784]: 2024-04-24 22:55:04.211946023 +0000 UTC m=+5.919050588 container cleanup bd3bce09c31c23f978bf3deeeff18c78af61b8983760e717d57fc34857a43d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:55:04.407 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:04 smithi171 bash[68784]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4 2024-04-24T22:55:04.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:04 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:04.302+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:04.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:04 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:04.302+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:04.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:04.351+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:04.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:04.351+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:04.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:04.564+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:04.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:04.564+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:04.896 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:04 smithi171 podman[68808]: 2024-04-24 22:55:04.538249285 +0000 UTC m=+0.680647780 container remove bd3bce09c31c23f978bf3deeeff18c78af61b8983760e717d57fc34857a43d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T22:55:05.059 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:04 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:04.784+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:05.059 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:04 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:04.784+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:05.156 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:05 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:05.080+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:05.157 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:04 smithi171 podman[68906]: 2024-04-24 22:55:04.798352304 +0000 UTC m=+0.018290098 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:05.157 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:04 smithi171 podman[68906]: 2024-04-24 22:55:04.89739918 +0000 UTC m=+0.117336958 container create d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:05.157 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 podman[68906]: 2024-04-24 22:55:05.124739035 +0000 UTC m=+0.344676814 container init d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T22:55:05.157 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 podman[68906]: 2024-04-24 22:55:05.128120063 +0000 UTC m=+0.348057845 container start d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:05.499 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:05 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:05.057+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:05.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:05 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:05.304+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:05.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:05 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:05.304+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:05.599 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:05 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:05.325+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:05.599 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:05 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:05.325+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:05.657 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 podman[68906]: 2024-04-24 22:55:05.216508375 +0000 UTC m=+0.436446166 container attach d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T22:55:05.657 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 podman[68906]: 2024-04-24 22:55:05.42055342 +0000 UTC m=+0.640491199 container died d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:55:05.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:05 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:05.350+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:05.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:05 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:05.350+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:05.940 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T22:55:05.964 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 podman[68932]: 2024-04-24 22:55:05.911449271 +0000 UTC m=+0.480258699 container remove d1c7d5902653096c65630613f26aef0083df9b7504638cd318ce656884992a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:55:05.964 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-24T22:55:05.964 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 systemd[1]: Stopped Ceph osd.4 for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:55:05.964 INFO:journalctl@ceph.osd.4.smithi171.stdout:Apr 24 22:55:05 smithi171 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.4.service: Consumed 11.531s CPU time. 2024-04-24T22:55:05.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:05.980 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T22:55:05.980 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T22:55:05.980 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.5 2024-04-24T22:55:06.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:05 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:05.597+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:06.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:05 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:05.597+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:06.089 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:05 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:05.830+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:06.089 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:05 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:05.830+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:06.089 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 systemd[1]: Stopping Ceph osd.5 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:06.347 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:06.087+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:06.347 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:06.277+0000 7f83d93cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:06.347 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:06.277+0000 7f83d93cf700 -1 osd.5 104 *** Got signal Terminated *** 2024-04-24T22:55:06.348 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:06.277+0000 7f83d93cf700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:06.353 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:06 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:06.128+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:06.590 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:06 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:06.287+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:06.590 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:06 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:06.287+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:06.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:06 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:06.352+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:06.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:06 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:06.352+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:06.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:06.346+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:06.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:06.346+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:07.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:06 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:06.588+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:07.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:06 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:06.588+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:07.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:07.070+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:07.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:06.839+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:07.249 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:06 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:06.839+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:07.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:07 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:07.164+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:07.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:07 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:07.310+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:07.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:07 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:07.310+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:07.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:07.358+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:07.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:07.358+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:07.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:07.358+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:07.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:07 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:07.576+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:07.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:07 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:07.576+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:07.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:07 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:07.338+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:07.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:07 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:07.338+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:07.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:07 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:07.338+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:08.084 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:07.799+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:08.084 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:07 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:07.799+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:08.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:08.082+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:08.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:08.082+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:08.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:08 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:08.168+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:08.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:08 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:08.168+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:08.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:08 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:08.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:08.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:08 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:08.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:08.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:08 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:08.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:08.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:08.359+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:08.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:08.359+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:08.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:08.359+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:08.380+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:08.380+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:08.380+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:08.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:08.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:08.786 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:08.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:09.042 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:08.765+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:09.042 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:08 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:08.765+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:09.348 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:09.041+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:09.348 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:09.041+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:09.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:09 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:09.132+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:09.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:09 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:09.132+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:09.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:09 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:09.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:09.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:09 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:09.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:09.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:09 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:09.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:09.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:09.347+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:09.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:09.347+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:09.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:09.347+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:09.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:09.384+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:09.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:09.384+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:09.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:09.384+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:09.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:09.621+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:09.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:09.621+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:09.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:09 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:09.621+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:10.085 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:09.769+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:10.085 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:09 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:09.769+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:10.380 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:10.084+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:10.381 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:10.084+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:10.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:10 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:10.120+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:10.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:10 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:10.120+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:10.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:10 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:10.267+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:10.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:10 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:10.267+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:10.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:10 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:10.267+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:10.649 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:10.351+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:10.649 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:10.351+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:10.649 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:10.351+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:10.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:10.379+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:10.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:10.379+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:10.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:10.379+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:11.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:10.647+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:11.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:10.647+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:11.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:10 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:10.647+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:11.101 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:10.773+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.141826+0000 front 2024-04-24T22:54:29.141618+0000 (oldest deadline 2024-04-24T22:54:54.341835+0000) 2024-04-24T22:55:11.101 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:10 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5[44911]: 2024-04-24T22:55:10.773+0000 7f83d61c4700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.642991+0000 front 2024-04-24T22:54:36.643277+0000 (oldest deadline 2024-04-24T22:55:02.542960+0000) 2024-04-24T22:55:11.399 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:11.099+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:11.399 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:11.099+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:11.658 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:11.158+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:11.658 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:11.158+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:11.659 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:11.247+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:11.659 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:11.247+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:11.659 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:11 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:11.247+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:11.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:11.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:11.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:11.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:11.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:11 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:11.397+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:11.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:11.385+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:11.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:11.385+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:11.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:11.385+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:11.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:11.607+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:11.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:11.607+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:11.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:11 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:11.607+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:12.094 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:11 smithi186 podman[70071]: 2024-04-24 22:55:11.805692368 +0000 UTC m=+5.634819707 container died b269345a00d4c9b5c1a91ee1f0cd5ec9493666c9c64003baf255841216387698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T22:55:12.400 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:12 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:12.093+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:12.401 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:12 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:12.093+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:12.401 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:12 smithi186 podman[70071]: 2024-04-24 22:55:12.19435533 +0000 UTC m=+6.023482656 container cleanup b269345a00d4c9b5c1a91ee1f0cd5ec9493666c9c64003baf255841216387698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T22:55:12.401 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:12 smithi186 bash[70071]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5 2024-04-24T22:55:12.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:12 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:12.184+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:12.657 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:12 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:12.184+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:12.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:12 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:12.263+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:12.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:12 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:12.263+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:12.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:12 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:12.263+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:12.678 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:12.403+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:12.679 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:12.403+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:12.679 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:12.403+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:12.679 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:12.562+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:12.679 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:12.562+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:12.679 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:12 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:12.562+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:12.749 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:12 smithi186 podman[70089]: 2024-04-24 22:55:12.4455356 +0000 UTC m=+0.631245424 container remove b269345a00d4c9b5c1a91ee1f0cd5ec9493666c9c64003baf255841216387698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T22:55:12.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:12 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:12.399+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:12.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:12 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:12.399+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:12.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:12 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:12.399+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:13.062 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:12 smithi186 podman[70185]: 2024-04-24 22:55:12.665020479 +0000 UTC m=+0.018134367 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:13.062 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:12 smithi186 podman[70185]: 2024-04-24 22:55:12.779619304 +0000 UTC m=+0.132733233 container create 312350defb8f602bc63f4b93eeaa49fd66578b4306e037c90b1b2415c972caa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:13.062 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:13 smithi186 podman[70185]: 2024-04-24 22:55:13.007424203 +0000 UTC m=+0.360538098 container init 312350defb8f602bc63f4b93eeaa49fd66578b4306e037c90b1b2415c972caa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T22:55:13.062 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:13 smithi186 podman[70185]: 2024-04-24 22:55:13.010680133 +0000 UTC m=+0.363794021 container start 312350defb8f602bc63f4b93eeaa49fd66578b4306e037c90b1b2415c972caa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T22:55:13.443 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:13 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:13.060+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:13.443 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:13 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:13.060+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:13.443 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:13 smithi186 podman[70185]: 2024-04-24 22:55:13.100732304 +0000 UTC m=+0.453846194 container attach 312350defb8f602bc63f4b93eeaa49fd66578b4306e037c90b1b2415c972caa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T22:55:13.443 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 24 22:55:13 smithi186 podman[70185]: 2024-04-24 22:55:13.305101552 +0000 UTC m=+0.658215439 container died 312350defb8f602bc63f4b93eeaa49fd66578b4306e037c90b1b2415c972caa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:55:13.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:13 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:13.165+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:13.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:13 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:13.165+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:13.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:13 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:13.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:13.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:13 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:13.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:13.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:13 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:13.254+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:13.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:13 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:13.441+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:13.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:13 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:13.441+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:13.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:13 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:13.441+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:13.364+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:13.364+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:13.364+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:13.557+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:13.557+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:13.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:13 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:13.557+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:13.860 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T22:55:13.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:13.897 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T22:55:13.897 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T22:55:13.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.6 2024-04-24T22:55:14.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:14.012+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:14.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:14.012+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:14.283 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 systemd[1]: Stopping Ceph osd.6 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:14.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:14 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:14.162+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:14.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:14 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:14.162+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:14.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:14 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:14.239+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:14.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:14 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:14.239+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:14.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:14 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:14.239+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:14.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:14.453+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:14.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:14.453+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:14.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:14.453+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:14.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:14.544+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:14.544+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:14.544+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.383+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.383+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.383+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.468+0000 7f11caa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.468+0000 7f11caa00700 -1 osd.6 104 *** Got signal Terminated *** 2024-04-24T22:55:14.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:14.468+0000 7f11caa00700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:15.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:14.978+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:15.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:14 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:14.978+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:15.263 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:15 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:15.145+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:15.264 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:15 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:15.145+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:15.264 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:15 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:15.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:15.264 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:15 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:15.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:15.264 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:15 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:15.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:15.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:15.442+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:15.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:15.442+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:15.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:15.442+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:15.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:15.442+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:15.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:15.350+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:15.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:15.350+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:15.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:15.350+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:15.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:15.560+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:15.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:15.560+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:15.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:15 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:15.560+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:16.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:15.967+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:16.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:15.967+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:16.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:15 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:15.967+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:16.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:16.165+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:16.165+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:16.165+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:16.278+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:16.278+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:16.278+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:16.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:16 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:16.278+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:16.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:16 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:16.431+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:16.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:16 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:16.431+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:16.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:16 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:16.431+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:16.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:16 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:16.431+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:16.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:16.305+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:16.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:16.305+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:16.786 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:16.305+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:16.786 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:16.513+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:16.786 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:16.513+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:16.787 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:16.513+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:16.787 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:16 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:16.513+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:17.398 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.004+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:17.399 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.004+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:17.399 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.004+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:17.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:17.215+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:17.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:17.215+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:17.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:17.215+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:17.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:17.232+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:17.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:17.232+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:17.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:17.232+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:17.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:17 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:17.232+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:17.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:17.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:17.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:17.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:17.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:17.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:17.751 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:17.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:17.340+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:17.340+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:17.340+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:17.340+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.037146+0000 front 2024-04-24T22:54:53.037250+0000 (oldest deadline 2024-04-24T22:55:16.537005+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:17.503+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:17.503+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:17.503+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:17.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:17.503+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:18.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.971+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:18.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.971+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:18.250 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:17 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:17.971+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:18.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:18.195+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:18.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:18.195+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:18.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:18.195+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:18.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:18.217+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:18.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:18.217+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:18.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:18.217+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:18.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:18 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:18.217+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:18.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:18.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:18.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:18.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:18.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:18.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:18.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:18.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:18.360+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:18.360+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:18.360+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:18.360+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.037146+0000 front 2024-04-24T22:54:53.037250+0000 (oldest deadline 2024-04-24T22:55:16.537005+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:18.529+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:18.529+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:18.529+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:18.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:18 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:18.529+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:19.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:18.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:19.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:18.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:19.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:18 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:18.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:19.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:19.231+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:19.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:19.231+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:19.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:19.231+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:19.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:19.225+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:19.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:19.225+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:19.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:19.225+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:19.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:19 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:19.225+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:19.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:19.371+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:19.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:19.371+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:19.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:19.371+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:19.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:19.371+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:19.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:19.312+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:27.930080+0000 front 2024-04-24T22:54:27.929333+0000 (oldest deadline 2024-04-24T22:54:53.229250+0000) 2024-04-24T22:55:19.783 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:19.312+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:36.731153+0000 front 2024-04-24T22:54:36.731213+0000 (oldest deadline 2024-04-24T22:54:59.031388+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:19.312+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:42.434879+0000 front 2024-04-24T22:54:42.434828+0000 (oldest deadline 2024-04-24T22:55:07.134902+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6[56312]: 2024-04-24T22:55:19.312+0000 7f11c1c00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.037146+0000 front 2024-04-24T22:54:53.037250+0000 (oldest deadline 2024-04-24T22:55:16.537005+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:19.510+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:19.510+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:19.510+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:19.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:19 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:19.510+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:20.152 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:19 smithi111 podman[89203]: 2024-04-24 22:55:19.828738669 +0000 UTC m=+5.506307100 container died e9304d1e0f46e724a5f177b48dfbbbd18d50c4dca2c1388450dc4aa7a2de831c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T22:55:20.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:19.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:20.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:19.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:20.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:19 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:19.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:20.533 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:20 smithi111 podman[89203]: 2024-04-24 22:55:20.269895314 +0000 UTC m=+5.947463782 container cleanup e9304d1e0f46e724a5f177b48dfbbbd18d50c4dca2c1388450dc4aa7a2de831c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:55:20.534 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:20 smithi111 bash[89203]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6 2024-04-24T22:55:20.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:20.219+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:20.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:20.219+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:20.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:20.219+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:20.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:20.269+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:20.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:20.269+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:20.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:20.269+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:20.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:20 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:20.269+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:20.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:20.335+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:20.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:20.335+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:20.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:20.335+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:20.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:20.335+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:21.033 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:20 smithi111 podman[89222]: 2024-04-24 22:55:20.549887534 +0000 UTC m=+0.708985898 container remove e9304d1e0f46e724a5f177b48dfbbbd18d50c4dca2c1388450dc4aa7a2de831c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T22:55:21.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:20.539+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:21.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:20.539+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:21.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:20.539+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:21.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:20 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:20.539+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:21.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:20.877+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:21.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:20.877+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:21.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:20 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:20.877+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:21.656 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:21.198+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:21.657 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:21.198+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:21.657 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:21.198+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:21.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:21.314+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:21.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:21.314+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:21.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:21.314+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:21.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:21 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:21.314+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:21.669 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:21 smithi111 podman[89310]: 2024-04-24 22:55:21.24697195 +0000 UTC m=+0.032976558 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:21.669 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:21 smithi111 podman[89310]: 2024-04-24 22:55:21.405251939 +0000 UTC m=+0.191256587 container create a9bae6b7370486303f575f5d700c83ead69a5876bde2285fffdfc2b1d4db7153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T22:55:21.669 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:21 smithi111 podman[89310]: 2024-04-24 22:55:21.667332092 +0000 UTC m=+0.453336747 container init a9bae6b7370486303f575f5d700c83ead69a5876bde2285fffdfc2b1d4db7153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T22:55:21.670 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:21.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:21.670 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:21.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:21.670 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:21.579+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:21.670 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:21 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:21.580+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:21.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:21.373+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:21.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:21.373+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:21.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:21.373+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:21.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:21.373+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:22.033 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:21 smithi111 podman[89310]: 2024-04-24 22:55:21.67825463 +0000 UTC m=+0.464259281 container start a9bae6b7370486303f575f5d700c83ead69a5876bde2285fffdfc2b1d4db7153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:55:22.033 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:21 smithi111 podman[89310]: 2024-04-24 22:55:21.757899901 +0000 UTC m=+0.543904609 container attach a9bae6b7370486303f575f5d700c83ead69a5876bde2285fffdfc2b1d4db7153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T22:55:22.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:21.890+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:22.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:21.890+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:22.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:21 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:21.890+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:22.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:22.154+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:22.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:22.154+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:22.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:22.154+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:22.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:22.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:22.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:22.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:22.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:22.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:22.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:22.284+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:22.532 INFO:journalctl@ceph.osd.6.smithi111.stdout:Apr 24 22:55:22 smithi111 podman[89310]: 2024-04-24 22:55:22.129878526 +0000 UTC m=+0.915883177 container died a9bae6b7370486303f575f5d700c83ead69a5876bde2285fffdfc2b1d4db7153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T22:55:22.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:22.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:22.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:22.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:22.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:22.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:22.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:22.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:22.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:22.612+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:22.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:22.612+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:22.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:22.612+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:22.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:22 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:22.612+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:22.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T22:55:23.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:23.086 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T22:55:23.086 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T22:55:23.086 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.7 2024-04-24T22:55:23.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:22.915+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:23.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:22.915+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:23.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:22 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:22.915+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:23.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.124+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:23.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.124+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:23.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.124+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:23.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 systemd[1]: Stopping Ceph osd.7 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:23.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:23.288+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:23.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:23.288+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:23.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:23.288+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:23.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:23.288+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:23.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:23.408+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:23.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:23.408+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:23.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:23.408+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:23.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:23.408+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:23.906 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.412+0000 7ffa1cd74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:23.906 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.413+0000 7ffa1cd74700 -1 osd.7 104 *** Got signal Terminated *** 2024-04-24T22:55:23.906 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:23 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:23.413+0000 7ffa1cd74700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:24.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:23 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:23.613+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:24.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:23 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:23.613+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:24.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:23 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:23.613+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:24.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:23 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:23.613+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:24.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:23.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:24.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:23.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:24.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:23 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:23.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:24.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:24.112+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:24.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:24.112+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:24.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:24.112+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:24.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:24.246+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:24.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:24.246+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:24.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:24.246+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:24.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:24.246+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:24.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:24.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:24.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:24.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:24.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:24.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:24.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:24.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:25.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:24.598+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:25.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:24.598+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:25.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:24.598+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:25.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:24 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:24.598+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:25.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:24.917+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:25.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:24.917+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:25.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:24.917+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:25.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:24 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:24.917+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:25.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:25.064+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:25.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:25.064+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:25.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:25.064+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:25.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:25.265+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:25.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:25.265+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:25.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:25.265+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:25.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:25 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:25.265+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:25.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:25.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:25.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:25.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:25.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:25.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:25.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:25.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:26.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:25.554+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:26.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:25.554+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:26.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:25.554+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:26.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:25 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:25.554+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:26.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:25.918+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:26.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:25.918+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:26.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:25.918+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:26.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:25 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:25.918+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:26.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:26.030+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:26.030+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:26.030+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:26.030+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:54:59.196450+0000 front 2024-04-24T22:54:59.196404+0000 (oldest deadline 2024-04-24T22:55:25.096604+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:26.233+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:26.233+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:26.233+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:26.233+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:26.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:26 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:26.233+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:26.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:26.417+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:26.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:26.417+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:26.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:26.417+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:26.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:26.417+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:26.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:26.417+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:27.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:26.573+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:27.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:26.573+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:27.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:26.573+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:27.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:26.573+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:27.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:26 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:26.573+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:27.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:26.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:27.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:26.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:27.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:26.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:27.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:26 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:26.950+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:27.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:27.019+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:27.019+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:27.019+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:27.019+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:54:59.196450+0000 front 2024-04-24T22:54:59.196404+0000 (oldest deadline 2024-04-24T22:55:25.096604+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:27.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:27.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:27.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:27.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:27.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:27 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:27.261+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:27.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:27.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:27.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:27.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:27.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:27.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:27.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:27.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:27.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:27.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:27.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:27.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:27.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:27.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:27.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:27.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:27.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:27.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:27.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:27 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:27.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:28.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:27.916+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:28.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:27.916+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:28.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:27.916+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:28.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:27 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:27.916+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:28.271 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:28.003+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:35.892737+0000 front 2024-04-24T22:54:35.892832+0000 (oldest deadline 2024-04-24T22:54:59.992875+0000) 2024-04-24T22:55:28.271 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:28.003+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:43.494010+0000 front 2024-04-24T22:54:43.493980+0000 (oldest deadline 2024-04-24T22:55:07.593839+0000) 2024-04-24T22:55:28.271 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:28.003+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.295367+0000 front 2024-04-24T22:54:52.295196+0000 (oldest deadline 2024-04-24T22:55:15.695644+0000) 2024-04-24T22:55:28.272 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7[50180]: 2024-04-24T22:55:28.003+0000 7ffa15b61700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:54:59.196450+0000 front 2024-04-24T22:54:59.196404+0000 (oldest deadline 2024-04-24T22:55:25.096604+0000) 2024-04-24T22:55:28.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:28.270+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:28.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:28.270+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:28.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:28.270+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:28.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:28.270+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:28.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:28.270+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:28.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:28.435+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:28.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:28.435+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:28.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:28.435+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:28.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:28.435+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:28.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:28.435+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:28.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:28 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:28.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:28.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:28 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:28.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:28.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:28 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:28.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:28.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:28 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:28.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:28.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:28 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:28.526+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:29.197 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:28 smithi171 podman[69011]: 2024-04-24 22:55:28.918075792 +0000 UTC m=+5.629399505 container died 92622334e2ff260fac947702098d66ece258ed8e5f4898632683afafac34b8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T22:55:29.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:28.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:29.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:28.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:29.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:28.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:29.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:28 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:28.957+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:29 smithi171 podman[69011]: 2024-04-24 22:55:29.296092828 +0000 UTC m=+6.007416533 container cleanup 92622334e2ff260fac947702098d66ece258ed8e5f4898632683afafac34b8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:29 smithi171 bash[69011]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:29 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:29.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:29 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:29.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:29 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:29.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:29 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:29.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:29.523 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:29 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:29.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:29.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:29.406+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:29.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:29.406+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:29.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:29.406+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:29.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:29.406+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:29.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:29.406+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:29.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:29.489+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:29.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:29.489+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:29.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:29.489+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:29.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:29.489+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:29.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:29 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:29.489+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:29.832 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:29 smithi171 podman[69028]: 2024-04-24 22:55:29.522033066 +0000 UTC m=+0.604092479 container remove 92622334e2ff260fac947702098d66ece258ed8e5f4898632683afafac34b8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T22:55:30.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:29 smithi171 podman[69121]: 2024-04-24 22:55:29.831170278 +0000 UTC m=+0.108065280 container create 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T22:55:30.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:29 smithi171 podman[69121]: 2024-04-24 22:55:29.741294331 +0000 UTC m=+0.018189348 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:30.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:30 smithi171 podman[69121]: 2024-04-24 22:55:30.041964246 +0000 UTC m=+0.318859250 container init 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T22:55:30.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:30 smithi171 podman[69121]: 2024-04-24 22:55:30.044974335 +0000 UTC m=+0.321869340 container start 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:30.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:30 smithi171 podman[69121]: 2024-04-24 22:55:30.125205978 +0000 UTC m=+0.402100988 container attach 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T22:55:30.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:29.937+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:30.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:29.937+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:30.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:29.937+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:30.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:29 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:29.937+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:30.406 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:30 smithi171 podman[69121]: 2024-04-24 22:55:30.33014663 +0000 UTC m=+0.607041628 container died 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T22:55:30.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:30 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:30.253+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:30.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:30 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:30.253+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:30.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:30 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:30.253+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:30.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:30 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:30.253+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:30.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:30 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:30.253+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:30.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:30.401+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:30.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:30.401+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:30.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:30.401+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:30.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:30.401+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:30.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:30.401+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:30.900 INFO:journalctl@ceph.osd.7.smithi171.stdout:Apr 24 22:55:30 smithi171 podman[69147]: 2024-04-24 22:55:30.886820543 +0000 UTC m=+0.546605693 container remove 81703359230c9a08302fe8b6db8e0f69d9ebcdd36ef67a0dbb53c678310c8265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T22:55:30.906 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T22:55:30.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:30.944 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T22:55:30.945 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T22:55:30.945 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.8 2024-04-24T22:55:31.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:30.535+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:31.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:30.535+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:31.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:30.535+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:31.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:30.535+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:31.034 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:30 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:30.535+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:31.247 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:30.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:31.247 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:30.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:31.247 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:30.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:31.248 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:30 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:30.903+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:31.248 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:30 smithi186 systemd[1]: Stopping Ceph osd.8 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:31.248 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.245+0000 7fc3281db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:31.248 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.245+0000 7fc3281db700 -1 osd.8 104 *** Got signal Terminated *** 2024-04-24T22:55:31.248 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.245+0000 7fc3281db700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:31.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:31.354+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:31.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:31.354+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:31.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:31.354+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:31.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:31.354+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:31.499 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:31.354+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:31.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:31 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:31.271+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:31.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:31 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:31.271+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:31.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:31 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:31.271+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:31.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:31 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:31.271+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:31.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:31 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:31.271+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:31.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:31 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:31.508+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:31.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:31 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:31.508+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:31.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:31 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:31.508+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:31.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:31 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:31.508+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:31.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:31 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:31.508+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:32.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.900+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:32.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.900+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:32.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.900+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:32.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:31 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:31.900+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:32.257 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:32.240+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:32.257 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:32.240+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:32.257 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:32.240+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:32.257 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:32.240+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:32.257 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:32.240+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:32.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:32.338+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:32.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:32.338+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:32.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:32.338+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:32.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:32.338+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:32.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:32.338+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:32.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:32.475+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:32.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:32.475+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:32.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:32.475+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:32.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:32.475+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:32.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:32 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:32.475+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:33.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:32.942+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:33.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:32.942+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:33.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:32.942+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:33.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:32.942+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:33.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:32 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:32.942+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:06.747563+0000 front 2024-04-24T22:55:06.747534+0000 (oldest deadline 2024-04-24T22:55:32.647720+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:33.558 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:33 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:33.228+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:33.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:33.293+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:33.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:33.466+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:33.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:33.466+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:33.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:33.466+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:33.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:33.466+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:33.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:33 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:33.466+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:34.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:33.902+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:34.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:33.902+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:34.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:33.902+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:34.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:33.902+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:34.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:33 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:33.902+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:06.747563+0000 front 2024-04-24T22:55:06.747534+0000 (oldest deadline 2024-04-24T22:55:32.647720+0000) 2024-04-24T22:55:34.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:34.426+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:34.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:34.426+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:34.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:34.426+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:34.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:34.426+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:34.534 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:34 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:34.426+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:34.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:34.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:34.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:34.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:34.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:34.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:34 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:34.198+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:34.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:34.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:34.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:34.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:34.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:34.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:34.274+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:35.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:34.930+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:35.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:34.930+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:35.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:34.930+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:35.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:34.930+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:35.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:34 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:34.930+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:06.747563+0000 front 2024-04-24T22:55:06.747534+0000 (oldest deadline 2024-04-24T22:55:32.647720+0000) 2024-04-24T22:55:35.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:35.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:35.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:35.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:35.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:35.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:35 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:35.169+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:35.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:35.290+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:35.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:35 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:35.465+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:36.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:35.882+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.841925+0000 front 2024-04-24T22:54:37.841869+0000 (oldest deadline 2024-04-24T22:55:01.342002+0000) 2024-04-24T22:55:36.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:35.882+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.342389+0000 front 2024-04-24T22:54:41.342295+0000 (oldest deadline 2024-04-24T22:55:07.242478+0000) 2024-04-24T22:55:36.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:35.882+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.844216+0000 front 2024-04-24T22:54:51.844107+0000 (oldest deadline 2024-04-24T22:55:15.343968+0000) 2024-04-24T22:55:36.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:35.882+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:03.346928+0000 front 2024-04-24T22:55:03.346799+0000 (oldest deadline 2024-04-24T22:55:24.446647+0000) 2024-04-24T22:55:36.249 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:35 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8[51928]: 2024-04-24T22:55:35.882+0000 7fc320fc8700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:06.747563+0000 front 2024-04-24T22:55:06.747534+0000 (oldest deadline 2024-04-24T22:55:32.647720+0000) 2024-04-24T22:55:36.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:36.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:36.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:36.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:36.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:36.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:36 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:36.203+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:36.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.326+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:36.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.327+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:36.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.327+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:36.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.327+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:36.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.327+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:36.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:36 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:36.327+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:36.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:36.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:36.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:36.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:36.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:36.785 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:36 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:36.432+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:37.080 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:36 smithi186 podman[70290]: 2024-04-24 22:55:36.788046487 +0000 UTC m=+5.645072805 container died 945875b321e454a425991fb7042a696ff19d30d45ffd610bf79d982fd966098b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:55:37.357 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:37 smithi186 podman[70290]: 2024-04-24 22:55:37.178562091 +0000 UTC m=+6.035588417 container cleanup 945875b321e454a425991fb7042a696ff19d30d45ffd610bf79d982fd966098b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:37.357 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:37 smithi186 bash[70290]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8 2024-04-24T22:55:37.657 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:37.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:37.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:37.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:37.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:37.658 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:37 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:37.165+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:37.749 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:37 smithi186 podman[70307]: 2024-04-24 22:55:37.438205982 +0000 UTC m=+0.642933841 container remove 945875b321e454a425991fb7042a696ff19d30d45ffd610bf79d982fd966098b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T22:55:37.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:37.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:37.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:37.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:37.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:37.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:37.355+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:37.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:37.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:37.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:37.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:37.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:37.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:37 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:37.409+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:38.100 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:37 smithi186 podman[70398]: 2024-04-24 22:55:37.695032497 +0000 UTC m=+0.017997457 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:38.100 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:37 smithi186 podman[70398]: 2024-04-24 22:55:37.805894233 +0000 UTC m=+0.128859178 container create 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T22:55:38.100 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:38 smithi186 podman[70398]: 2024-04-24 22:55:38.033303856 +0000 UTC m=+0.356268801 container init 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T22:55:38.101 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:38 smithi186 podman[70398]: 2024-04-24 22:55:38.036990479 +0000 UTC m=+0.359955425 container start 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T22:55:38.101 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:38 smithi186 podman[70398]: 2024-04-24 22:55:38.099717854 +0000 UTC m=+0.422682818 container attach 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:55:38.381 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:38 smithi186 podman[70398]: 2024-04-24 22:55:38.341548557 +0000 UTC m=+0.664513495 container died 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:55:38.381 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:38.381 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:38.381 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:38.381 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:38.382 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:38.382 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:38 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:38.325+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:38.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:38.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:38.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:38.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:38.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:38.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:38 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:38.127+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:38.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:38 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:38.456+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:38.899 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 24 22:55:38 smithi186 podman[70425]: 2024-04-24 22:55:38.87938225 +0000 UTC m=+0.532202013 container remove 9961633eed8c202108e28434228ab82062cbbccc9eec52e1e0e0924a1c60eacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T22:55:38.903 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T22:55:38.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:38.940 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T22:55:38.940 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T22:55:38.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.9 2024-04-24T22:55:39.283 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 systemd[1]: Stopping Ceph osd.9 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:39.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:39.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:39.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:39.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:39.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:39.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:39 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:39.100+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:39.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:39 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:39.361+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.425+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.528+0000 7fa1ed600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.528+0000 7fa1ed600700 -1 osd.9 104 *** Got signal Terminated *** 2024-04-24T22:55:39.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:39 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:39.528+0000 7fa1ed600700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:40.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:40.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:40.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:40.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:40.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:40.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:40 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:40.059+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:40.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:40.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:40.783 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:40.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:40.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:40.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:40.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:40.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:40 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:40.423+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:41.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:41.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:41.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:41.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:41.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:41.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:41 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:41.035+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:41.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:41.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:41.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:41.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:41.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:41.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:41 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:41.428+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:41.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:41 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:41.445+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:15.365697+0000 front 2024-04-24T22:55:15.365893+0000 (oldest deadline 2024-04-24T22:55:40.665691+0000) 2024-04-24T22:55:42.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:42.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:42 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:42.037+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:42.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:42 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:42.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:42.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:42 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:42.427+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:15.365697+0000 front 2024-04-24T22:55:15.365893+0000 (oldest deadline 2024-04-24T22:55:40.665691+0000) 2024-04-24T22:55:43.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:43.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:43.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:43.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:43.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:43.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:43.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:43 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:43.039+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:43.733 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:43 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:43.420+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:15.365697+0000 front 2024-04-24T22:55:15.365893+0000 (oldest deadline 2024-04-24T22:55:40.665691+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:43.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:43 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:43.464+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:44.410 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:44.411 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:44 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:44.077+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:24.849297+0000 front 2024-04-24T22:54:24.848599+0000 (oldest deadline 2024-04-24T22:54:50.748701+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:37.751741+0000 front 2024-04-24T22:54:37.751706+0000 (oldest deadline 2024-04-24T22:55:01.851730+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:44.153529+0000 front 2024-04-24T22:54:44.153443+0000 (oldest deadline 2024-04-24T22:55:08.253624+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:51.656809+0000 front 2024-04-24T22:54:51.656566+0000 (oldest deadline 2024-04-24T22:55:15.756180+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.359661+0000 front 2024-04-24T22:55:00.359716+0000 (oldest deadline 2024-04-24T22:55:25.659607+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.463077+0000 front 2024-04-24T22:55:08.463182+0000 (oldest deadline 2024-04-24T22:55:34.863892+0000) 2024-04-24T22:55:44.784 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9[62948]: 2024-04-24T22:55:44.373+0000 7fa1e9800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:15.365697+0000 front 2024-04-24T22:55:15.365893+0000 (oldest deadline 2024-04-24T22:55:40.665691+0000) 2024-04-24T22:55:44.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:44.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:44.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:44.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:45.000 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:45.000 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:44.501+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:45.245 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:44 smithi111 podman[89425]: 2024-04-24 22:55:44.947189933 +0000 UTC m=+5.577012685 container died 76f13029e15f8f072bd4d61bbce74141cc520a79bf8b164a5c1e04f0cf617be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T22:55:45.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:45.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:45.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:45.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:45.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:45.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:45.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:45 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:45.114+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:45.533 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:45 smithi111 podman[89425]: 2024-04-24 22:55:45.363600731 +0000 UTC m=+5.993423490 container cleanup 76f13029e15f8f072bd4d61bbce74141cc520a79bf8b164a5c1e04f0cf617be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T22:55:45.533 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:45 smithi111 bash[89425]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:45.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:45 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:45.499+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:46.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:45 smithi111 podman[89443]: 2024-04-24 22:55:45.656634187 +0000 UTC m=+0.688831678 container remove 76f13029e15f8f072bd4d61bbce74141cc520a79bf8b164a5c1e04f0cf617be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:55:46.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:46.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:46.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:46.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:46.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:46.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:46.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:46 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:46.115+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:46.746 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:46 smithi111 podman[89534]: 2024-04-24 22:55:46.300494303 +0000 UTC m=+0.034084959 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:46.746 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:46 smithi111 podman[89534]: 2024-04-24 22:55:46.445541015 +0000 UTC m=+0.179131711 container create 8972cf9baaa85db2a8916d3338d81c955745288edc6d8eca1cd4d02160f30b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T22:55:46.746 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:46 smithi111 podman[89534]: 2024-04-24 22:55:46.744477409 +0000 UTC m=+0.478068108 container init 8972cf9baaa85db2a8916d3338d81c955745288edc6d8eca1cd4d02160f30b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:46.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:46 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:46.503+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:47.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:46 smithi111 podman[89534]: 2024-04-24 22:55:46.757539709 +0000 UTC m=+0.491130381 container start 8972cf9baaa85db2a8916d3338d81c955745288edc6d8eca1cd4d02160f30b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-24T22:55:47.033 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:46 smithi111 podman[89534]: 2024-04-24 22:55:46.835007942 +0000 UTC m=+0.568598617 container attach 8972cf9baaa85db2a8916d3338d81c955745288edc6d8eca1cd4d02160f30b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T22:55:47.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:47.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:47.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:47.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:47.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:47.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:47.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:47 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:47.129+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:47.533 INFO:journalctl@ceph.osd.9.smithi111.stdout:Apr 24 22:55:47 smithi111 podman[89534]: 2024-04-24 22:55:47.205137188 +0000 UTC m=+0.938727903 container died 8972cf9baaa85db2a8916d3338d81c955745288edc6d8eca1cd4d02160f30b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T22:55:47.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:47.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:47 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:47.498+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:48.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:48.054 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T22:55:48.055 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T22:55:48.055 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.10 2024-04-24T22:55:48.356 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 systemd[1]: Stopping Ceph osd.10 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.128+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:48.357 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.355+0000 7f907c807700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:48.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.355+0000 7f907c807700 -1 osd.10 104 *** Got signal Terminated *** 2024-04-24T22:55:48.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:48 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:48.355+0000 7f907c807700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:48.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:48.454+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:49.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:49 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:49.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:49.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:49 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:49.412+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:50.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:50.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:50.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:50.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:50.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:50.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:50.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:50 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:50.081+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:50.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:50 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:50.410+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:51.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:51.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:51.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:51.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:51.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:51.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:51.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:51 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:51.056+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:51.699 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:51.700 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:51.700 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:51.700 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:51.700 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:51.700 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:51 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:51.396+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:52.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:52 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:52.105+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:52.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:52.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:52.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:52.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:52.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:52.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:52 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:52.413+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:53.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:25.918479+0000 front 2024-04-24T22:54:25.918441+0000 (oldest deadline 2024-04-24T22:54:50.018146+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:38.120479+0000 front 2024-04-24T22:54:38.120511+0000 (oldest deadline 2024-04-24T22:55:01.620638+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.621204+0000 front 2024-04-24T22:54:41.621169+0000 (oldest deadline 2024-04-24T22:55:08.021779+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:53.323227+0000 front 2024-04-24T22:54:53.323381+0000 (oldest deadline 2024-04-24T22:55:15.622840+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.124901+0000 front 2024-04-24T22:55:00.124959+0000 (oldest deadline 2024-04-24T22:55:25.424949+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:08.325886+0000 front 2024-04-24T22:55:08.325790+0000 (oldest deadline 2024-04-24T22:55:32.425680+0000) 2024-04-24T22:55:53.407 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10[56920]: 2024-04-24T22:55:53.078+0000 7f90795fc700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-04-24T22:55:18.827200+0000 front 2024-04-24T22:55:18.827115+0000 (oldest deadline 2024-04-24T22:55:41.126737+0000) 2024-04-24T22:55:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:53.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:53.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:53 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:53.448+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:53.991 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:53 smithi171 podman[69237]: 2024-04-24 22:55:53.706884155 +0000 UTC m=+5.463839460 container died fb3a5439d122e940f7085fc1cd5e107aa62ea027c7d143da1345adbf30c5dedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:55:54.331 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69237]: 2024-04-24 22:55:54.088339503 +0000 UTC m=+5.845294799 container cleanup fb3a5439d122e940f7085fc1cd5e107aa62ea027c7d143da1345adbf30c5dedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T22:55:54.331 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 bash[69237]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10 2024-04-24T22:55:54.656 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69254]: 2024-04-24 22:55:54.330960181 +0000 UTC m=+0.622511299 container remove fb3a5439d122e940f7085fc1cd5e107aa62ea027c7d143da1345adbf30c5dedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:54.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:54.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:54 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:54.444+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:54.968 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69346]: 2024-04-24 22:55:54.558689386 +0000 UTC m=+0.017140202 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:55:54.968 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69346]: 2024-04-24 22:55:54.656551836 +0000 UTC m=+0.115002624 container create d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T22:55:54.968 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69346]: 2024-04-24 22:55:54.884170545 +0000 UTC m=+0.342621342 container init d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:55:54.968 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69346]: 2024-04-24 22:55:54.887769696 +0000 UTC m=+0.346220484 container start d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-24T22:55:54.968 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:54 smithi171 podman[69346]: 2024-04-24 22:55:54.967048864 +0000 UTC m=+0.425499651 container attach d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:55:55.406 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:55 smithi171 podman[69346]: 2024-04-24 22:55:55.178818411 +0000 UTC m=+0.637269197 container died d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T22:55:55.682 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:55 smithi171 podman[69372]: 2024-04-24 22:55:55.653423424 +0000 UTC m=+0.466253965 container remove d562499637d84b1f0451c898a7115dbcf08aad6dad7fcfa6f7387bbd94bdec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T22:55:55.683 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:55 smithi171 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-24T22:55:55.683 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:55 smithi171 systemd[1]: Stopped Ceph osd.10 for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:55:55.683 INFO:journalctl@ceph.osd.10.smithi171.stdout:Apr 24 22:55:55 smithi171 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.10.service: Consumed 12.986s CPU time. 2024-04-24T22:55:55.684 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T22:55:55.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:55:55.722 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T22:55:55.722 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T22:55:55.723 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.11 2024-04-24T22:55:55.748 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:55.748 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:55.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:55.493+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:55.999 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:55 smithi186 systemd[1]: Stopping Ceph osd.11 for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:55:56.460 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.023+0000 7faace9b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:55:56.460 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.023+0000 7faace9b7700 -1 osd.11 104 *** Got signal Terminated *** 2024-04-24T22:55:56.460 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.023+0000 7faace9b7700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:55:56.460 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:56.460 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:56.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:56 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:56.458+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:57.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:57.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:57.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:57.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:57 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:57.482+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-04-24T22:55:35.468561+0000 front 2024-04-24T22:55:35.468591+0000 (oldest deadline 2024-04-24T22:55:57.168628+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:58.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:58.751 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:58 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:58.473+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-04-24T22:55:35.468561+0000 front 2024-04-24T22:55:35.468591+0000 (oldest deadline 2024-04-24T22:55:57.168628+0000) 2024-04-24T22:55:59.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:55:59.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:55:59 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:55:59.429+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-04-24T22:55:35.468561+0000 front 2024-04-24T22:55:35.468591+0000 (oldest deadline 2024-04-24T22:55:57.168628+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-04-24T22:54:29.059541+0000 front 2024-04-24T22:54:29.059358+0000 (oldest deadline 2024-04-24T22:54:53.159229+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-04-24T22:54:34.260630+0000 front 2024-04-24T22:54:34.260483+0000 (oldest deadline 2024-04-24T22:54:58.960507+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-24T22:54:41.861285+0000 front 2024-04-24T22:54:41.861261+0000 (oldest deadline 2024-04-24T22:55:07.061820+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-04-24T22:54:52.263389+0000 front 2024-04-24T22:54:52.263459+0000 (oldest deadline 2024-04-24T22:55:15.163224+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-04-24T22:55:00.464443+0000 front 2024-04-24T22:55:00.464369+0000 (oldest deadline 2024-04-24T22:55:25.764235+0000) 2024-04-24T22:56:00.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-24T22:55:09.265109+0000 front 2024-04-24T22:55:09.265134+0000 (oldest deadline 2024-04-24T22:55:32.765137+0000) 2024-04-24T22:56:00.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-04-24T22:55:27.367575+0000 front 2024-04-24T22:55:27.367533+0000 (oldest deadline 2024-04-24T22:55:54.367803+0000) 2024-04-24T22:56:00.750 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:00 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11[58541]: 2024-04-24T22:56:00.404+0000 7faac77a4700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-04-24T22:55:35.468561+0000 front 2024-04-24T22:55:35.468591+0000 (oldest deadline 2024-04-24T22:55:57.168628+0000) 2024-04-24T22:56:01.667 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:01 smithi186 podman[70518]: 2024-04-24 22:56:01.350115733 +0000 UTC m=+5.432046687 container died cf8a8626f928aaccc2bac4f6bfc0d36f0804a65e24ea12f6b4c83a40c005a3e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T22:56:01.967 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:01 smithi186 podman[70518]: 2024-04-24 22:56:01.748492497 +0000 UTC m=+5.830423451 container cleanup cf8a8626f928aaccc2bac4f6bfc0d36f0804a65e24ea12f6b4c83a40c005a3e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:56:01.967 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:01 smithi186 bash[70518]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11 2024-04-24T22:56:02.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:01 smithi186 podman[70535]: 2024-04-24 22:56:01.96626021 +0000 UTC m=+0.607348996 container remove cf8a8626f928aaccc2bac4f6bfc0d36f0804a65e24ea12f6b4c83a40c005a3e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T22:56:02.520 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.160491578 +0000 UTC m=+0.018318413 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:56:02.520 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.283660336 +0000 UTC m=+0.141487193 container create aad24eb6ff5585c23074d236767a30aa7044214625ab90b35a08bc0fbeae89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:56:02.819 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.519646013 +0000 UTC m=+0.377472823 container init aad24eb6ff5585c23074d236767a30aa7044214625ab90b35a08bc0fbeae89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T22:56:02.819 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.523044746 +0000 UTC m=+0.380871558 container start aad24eb6ff5585c23074d236767a30aa7044214625ab90b35a08bc0fbeae89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T22:56:02.819 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.611635277 +0000 UTC m=+0.469462104 container attach aad24eb6ff5585c23074d236767a30aa7044214625ab90b35a08bc0fbeae89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-24T22:56:03.114 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 24 22:56:02 smithi186 podman[70624]: 2024-04-24 22:56:02.818243021 +0000 UTC m=+0.676069844 container died aad24eb6ff5585c23074d236767a30aa7044214625ab90b35a08bc0fbeae89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T22:56:03.418 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T22:56:03.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:03.496 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T22:56:03.497 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T22:56:03.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.a 2024-04-24T22:56:04.024 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:03 smithi111 systemd[1]: Stopping Ceph mds.a for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:04.283 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a[70052]: 2024-04-24T22:56:04.021+0000 7f3adb200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:04.283 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:04 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a[70052]: 2024-04-24T22:56:04.021+0000 7f3adb200700 -1 mds.a *** got signal Terminated *** 2024-04-24T22:56:06.991 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:06 smithi111 podman[89651]: 2024-04-24 22:56:06.73045575 +0000 UTC m=+2.837388555 container died d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T22:56:07.283 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:07 smithi111 podman[89651]: 2024-04-24 22:56:07.115343466 +0000 UTC m=+3.222276300 container cleanup d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T22:56:07.284 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:07 smithi111 bash[89651]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a 2024-04-24T22:56:07.762 INFO:journalctl@ceph.mds.a.smithi111.stdout:Apr 24 22:56:07 smithi111 podman[89677]: 2024-04-24 22:56:07.316177933 +0000 UTC m=+0.567695920 container remove d19031486cb811649c56c30a6a96f5e78729ee75677d71f54f9dd641de48503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T22:56:07.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T22:56:07.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:07.910 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T22:56:07.910 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T22:56:07.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.d 2024-04-24T22:56:08.283 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:08 smithi111 systemd[1]: Stopping Ceph mds.d for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:08.783 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d[69050]: 2024-04-24T22:56:08.466+0000 7ff69f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:08.783 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:08 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d[69050]: 2024-04-24T22:56:08.466+0000 7ff69f000700 -1 mds.d *** got signal Terminated *** 2024-04-24T22:56:12.783 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:12 smithi111 podman[89782]: 2024-04-24 22:56:12.48516232 +0000 UTC m=+4.159492820 container died ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T22:56:13.112 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:12 smithi111 podman[89782]: 2024-04-24 22:56:12.901864324 +0000 UTC m=+4.576194817 container cleanup ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T22:56:13.112 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:12 smithi111 bash[89782]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d 2024-04-24T22:56:13.533 INFO:journalctl@ceph.mds.d.smithi111.stdout:Apr 24 22:56:13 smithi111 podman[89796]: 2024-04-24 22:56:13.111155164 +0000 UTC m=+0.608594208 container remove ea30de002adeba64bb78b0cc807a8f8b1e72b6c67eb658d6a67205a2e0962aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:56:13.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T22:56:13.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:13.823 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T22:56:13.823 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T22:56:13.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.g 2024-04-24T22:56:14.283 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:13 smithi111 systemd[1]: Stopping Ceph mds.g for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:14.784 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g[69659]: 2024-04-24T22:56:14.469+0000 7ff0e5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:14.784 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:14 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g[69659]: 2024-04-24T22:56:14.469+0000 7ff0e5400700 -1 mds.g *** got signal Terminated *** 2024-04-24T22:56:15.835 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:15 smithi111 podman[89900]: 2024-04-24 22:56:15.582409044 +0000 UTC m=+1.254036507 container died 533bcc9d208df6258cd745176bc63c4a5de354f2e7c5a3eca13f747c3a8791a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T22:56:16.191 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:15 smithi111 podman[89900]: 2024-04-24 22:56:15.971892303 +0000 UTC m=+1.643519794 container cleanup 533bcc9d208df6258cd745176bc63c4a5de354f2e7c5a3eca13f747c3a8791a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T22:56:16.192 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:15 smithi111 bash[89900]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g 2024-04-24T22:56:16.533 INFO:journalctl@ceph.mds.g.smithi111.stdout:Apr 24 22:56:16 smithi111 podman[89914]: 2024-04-24 22:56:16.189552415 +0000 UTC m=+0.590931542 container remove 533bcc9d208df6258cd745176bc63c4a5de354f2e7c5a3eca13f747c3a8791a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:56:16.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T22:56:16.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:16.788 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T22:56:16.788 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T22:56:16.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.j 2024-04-24T22:56:17.283 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:16 smithi111 systemd[1]: Stopping Ceph mds.j for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:17.783 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j[68702]: 2024-04-24T22:56:17.323+0000 7fb739400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:17.783 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:17 smithi111 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j[68702]: 2024-04-24T22:56:17.323+0000 7fb739400700 -1 mds.j *** got signal Terminated *** 2024-04-24T22:56:21.081 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:20 smithi111 podman[90018]: 2024-04-24 22:56:20.810338975 +0000 UTC m=+3.609610510 container died 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T22:56:21.386 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:21 smithi111 podman[90018]: 2024-04-24 22:56:21.192321077 +0000 UTC m=+3.991592622 container cleanup 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:56:21.387 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:21 smithi111 bash[90018]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j 2024-04-24T22:56:21.783 INFO:journalctl@ceph.mds.j.smithi111.stdout:Apr 24 22:56:21 smithi111 podman[90032]: 2024-04-24 22:56:21.384887588 +0000 UTC m=+0.556214290 container remove 6bcaa13fd6c266d0855900d8ac0f08820130561cc75e92aab3448c7c6de0220a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T22:56:21.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T22:56:22.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:22.021 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T22:56:22.021 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T22:56:22.021 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.b 2024-04-24T22:56:22.306 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:22 smithi171 systemd[1]: Stopping Ceph mds.b for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:22.656 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: 2024-04-24T22:56:22.323+0000 7f8cb19a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:22.656 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:22 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b[62163]: 2024-04-24T22:56:22.323+0000 7f8cb19a4700 -1 mds.b *** got signal Terminated *** 2024-04-24T22:56:23.812 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:23 smithi171 podman[69453]: 2024-04-24 22:56:23.524493558 +0000 UTC m=+1.313770300 container died 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T22:56:24.130 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:23 smithi171 podman[69453]: 2024-04-24 22:56:23.911481878 +0000 UTC m=+1.700758619 container cleanup 07004eddc28a72ca4decfce1d1700de3a6e6cd1a2c9e15d0e288b7527bc8b425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T22:56:24.130 INFO:journalctl@ceph.mds.b.smithi171.stdout:Apr 24 22:56:23 smithi171 bash[69453]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-b 2024-04-24T22:56:24.256 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T22:56:24.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:24.291 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T22:56:24.291 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T22:56:24.291 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.e 2024-04-24T22:56:24.407 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:24 smithi171 systemd[1]: Stopping Ceph mds.e for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:24.906 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e[61540]: 2024-04-24T22:56:24.598+0000 7f1a29a8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:24.906 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:24 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e[61540]: 2024-04-24T22:56:24.598+0000 7f1a29a8d700 -1 mds.e *** got signal Terminated *** 2024-04-24T22:56:27.899 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:27 smithi171 podman[69585]: 2024-04-24 22:56:27.636423041 +0000 UTC m=+3.134514856 container died b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T22:56:28.156 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:27 smithi171 podman[69585]: 2024-04-24 22:56:27.989814411 +0000 UTC m=+3.487906223 container cleanup b24210f614ad1c2ffb0e2792425704c0b69fe4486c4899ad45d7fc76d588d4e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T22:56:28.156 INFO:journalctl@ceph.mds.e.smithi171.stdout:Apr 24 22:56:27 smithi171 bash[69585]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-e 2024-04-24T22:56:28.413 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T22:56:28.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:28.446 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T22:56:28.446 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T22:56:28.446 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.h 2024-04-24T22:56:28.906 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:28 smithi171 systemd[1]: Stopping Ceph mds.h for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:28.906 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: 2024-04-24T22:56:28.759+0000 7f822eb72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:28.906 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:28 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h[61852]: 2024-04-24T22:56:28.759+0000 7f822eb72700 -1 mds.h *** got signal Terminated *** 2024-04-24T22:56:31.811 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:31 smithi171 podman[69708]: 2024-04-24 22:56:31.524006647 +0000 UTC m=+2.867842762 container died 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:56:32.145 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:31 smithi171 podman[69708]: 2024-04-24 22:56:31.893218504 +0000 UTC m=+3.237054616 container cleanup 707acaf489e421628d628c1eb733ac5d8e617354de92bb480f126331f1ecb1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:56:32.145 INFO:journalctl@ceph.mds.h.smithi171.stdout:Apr 24 22:56:31 smithi171 bash[69708]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-h 2024-04-24T22:56:32.266 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T22:56:32.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:32.299 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T22:56:32.299 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T22:56:32.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.k 2024-04-24T22:56:32.406 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:32 smithi171 systemd[1]: Stopping Ceph mds.k for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:32.906 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k[61215]: 2024-04-24T22:56:32.621+0000 7fe350a39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:32.906 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:32 smithi171 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k[61215]: 2024-04-24T22:56:32.621+0000 7fe350a39700 -1 mds.k *** got signal Terminated *** 2024-04-24T22:56:36.906 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:36 smithi171 podman[69830]: 2024-04-24 22:56:36.627337846 +0000 UTC m=+4.116186608 container died 61cee8c934343539d754248254ee8aec351cfc5f702b81f7cd57f423c07e76dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-24T22:56:37.239 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:37 smithi171 podman[69830]: 2024-04-24 22:56:37.071329919 +0000 UTC m=+4.560178680 container cleanup 61cee8c934343539d754248254ee8aec351cfc5f702b81f7cd57f423c07e76dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:56:37.239 INFO:journalctl@ceph.mds.k.smithi171.stdout:Apr 24 22:56:37 smithi171 bash[69830]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-k 2024-04-24T22:56:37.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T22:56:37.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:37.479 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T22:56:37.479 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T22:56:37.480 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.c 2024-04-24T22:56:37.773 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:37 smithi186 systemd[1]: Stopping Ceph mds.c for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:38.037 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: 2024-04-24T22:56:37.772+0000 7fcd0021f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:38.037 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:37 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c[63727]: 2024-04-24T22:56:37.772+0000 7fcd0021f700 -1 mds.c *** got signal Terminated *** 2024-04-24T22:56:39.890 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:39 smithi186 podman[70729]: 2024-04-24 22:56:39.525713471 +0000 UTC m=+1.859757537 container died 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T22:56:40.249 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:39 smithi186 podman[70729]: 2024-04-24 22:56:39.997665964 +0000 UTC m=+2.331710040 container cleanup 0a5408d3555130aa4131c99e16c1f5f98a179e5adb3344aecaf6bae377f8c8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:56:40.249 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 24 22:56:39 smithi186 bash[70729]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-c 2024-04-24T22:56:40.475 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T22:56:40.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:40.509 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T22:56:40.509 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T22:56:40.509 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.f 2024-04-24T22:56:40.836 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:40 smithi186 systemd[1]: Stopping Ceph mds.f for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:41.249 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: 2024-04-24T22:56:40.834+0000 7fe037c92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:41.249 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:40 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f[63406]: 2024-04-24T22:56:40.834+0000 7fe037c92700 -1 mds.f *** got signal Terminated *** 2024-04-24T22:56:43.786 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:43 smithi186 podman[70861]: 2024-04-24 22:56:43.520366571 +0000 UTC m=+2.795620227 container died 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True) 2024-04-24T22:56:44.195 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:43 smithi186 podman[70861]: 2024-04-24 22:56:43.876563969 +0000 UTC m=+3.151817625 container cleanup 43e7a9537ce596112486d6c3a0da4ef3e688c8ce5cc01ba3c286b44cee2eb0dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T22:56:44.195 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 24 22:56:43 smithi186 bash[70861]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-f 2024-04-24T22:56:44.350 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T22:56:44.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:44.381 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T22:56:44.381 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T22:56:44.381 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.i 2024-04-24T22:56:44.749 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:44 smithi186 systemd[1]: Stopping Ceph mds.i for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:44.749 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: 2024-04-24T22:56:44.688+0000 7f1482e8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:44.749 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:44 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i[64035]: 2024-04-24T22:56:44.688+0000 7f1482e8c700 -1 mds.i *** got signal Terminated *** 2024-04-24T22:56:47.857 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:47 smithi186 podman[70981]: 2024-04-24 22:56:47.526961155 +0000 UTC m=+2.931953692 container died 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:56:48.124 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:47 smithi186 podman[70981]: 2024-04-24 22:56:47.947629335 +0000 UTC m=+3.352621911 container cleanup 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:56:48.124 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:47 smithi186 bash[70981]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i 2024-04-24T22:56:48.367 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T22:56:48.387 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:48 smithi186 podman[70995]: 2024-04-24 22:56:48.123745309 +0000 UTC m=+0.599285114 container remove 57e6ca4126c80ce0e639d07f5e9d1f8e1a2c6365deff47be3f2a5cb0623b6455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T22:56:48.387 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:48 smithi186 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-24T22:56:48.387 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:48 smithi186 systemd[1]: Stopped Ceph mds.i for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:56:48.387 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 24 22:56:48 smithi186 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.i.service: Consumed 2.056s CPU time. 2024-04-24T22:56:48.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:48.399 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T22:56:48.399 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T22:56:48.399 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.l 2024-04-24T22:56:48.727 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:48 smithi186 systemd[1]: Stopping Ceph mds.l for 0d3ba9b8-028a-11ef-bc93-c7b262605968... 2024-04-24T22:56:48.999 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l[63098]: 2024-04-24T22:56:48.725+0000 7f6c536fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:56:48.999 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:48 smithi186 ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l[63098]: 2024-04-24T22:56:48.725+0000 7f6c536fa700 -1 mds.l *** got signal Terminated *** 2024-04-24T22:56:50.353 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 podman[71101]: 2024-04-24 22:56:50.09481795 +0000 UTC m=+1.474214209 container died 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T22:56:50.661 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 podman[71101]: 2024-04-24 22:56:50.434524995 +0000 UTC m=+1.813921262 container cleanup 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T22:56:50.661 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 bash[71101]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l 2024-04-24T22:56:50.902 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T22:56:50.922 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 podman[71115]: 2024-04-24 22:56:50.660665027 +0000 UTC m=+0.562449024 container remove 39fcf77a2abb65e26c713aaf0b9030615515afdb0e56fe0f5f29de3e7a4ea355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T22:56:50.922 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-24T22:56:50.922 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 systemd[1]: Stopped Ceph mds.l for 0d3ba9b8-028a-11ef-bc93-c7b262605968. 2024-04-24T22:56:50.922 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 24 22:56:50 smithi186 systemd[1]: ceph-0d3ba9b8-028a-11ef-bc93-c7b262605968@mds.l.service: Consumed 1.103s CPU time. 2024-04-24T22:56:50.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T22:56:50.933 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T22:56:50.933 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T22:56:51.256 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:56:56.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T22:56:56.311 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:56:58.832 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T22:56:59.022 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:57:01.339 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:57:01.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:57:01.460 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:57:01.487 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T22:57:01.489 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi111/crash 2024-04-24T22:57:01.490 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T22:57:01.589 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T22:57:01.589 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-04-24T22:57:01.594 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi171/crash 2024-04-24T22:57:01.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T22:57:01.642 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T22:57:01.642 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-04-24T22:57:01.644 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi186/crash 2024-04-24T22:57:01.645 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T22:57:01.681 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T22:57:01.681 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-04-24T22:57:01.682 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T22:57:01.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-24T22:57:01.792 INFO:tasks.cephadm:Compressing logs... 2024-04-24T22:57:01.792 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T22:57:01.797 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T22:57:01.799 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T22:57:01.822 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T22:57:01.822 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2024-04-24T22:57:01.822 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T22:57:01.823 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T22:57:01.824 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T22:57:01.824 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T22:57:01.824 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T22:57:01.824 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T22:57:01.827 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T22:57:01.827 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T22:57:01.827 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T22:57:01.827 INFO:teuthology.orchestra.run.smithi186.stderr: 90.3% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T22:57:01.831 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T22:57:01.831 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 87.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr: --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr: 90.2% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.3% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T22:57:01.832 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T22:57:01.834 INFO:teuthology.orchestra.run.smithi186.stderr: 92.1% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T22:57:01.835 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T22:57:01.836 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T22:57:01.836 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T22:57:01.836 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.11.log: 83.3% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T22:57:01.838 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T22:57:01.838 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T22:57:01.838 INFO:teuthology.orchestra.run.smithi171.stderr: 92.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log: 2024-04-24T22:57:01.847 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T22:57:01.848 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T22:57:01.850 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T22:57:01.850 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T22:57:01.852 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T22:57:01.853 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T22:57:01.853 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T22:57:01.854 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T22:57:01.855 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T22:57:01.856 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T22:57:01.857 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T22:57:01.857 INFO:teuthology.orchestra.run.smithi111.stderr: 87.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T22:57:01.857 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T22:57:01.858 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.j.log: 90.1% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T22:57:01.858 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T22:57:01.859 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T22:57:01.861 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T22:57:01.861 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T22:57:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- 94.6% /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.6.log -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T22:57:01.862 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-04-24T22:57:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T22:57:01.865 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T22:57:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T22:57:01.867 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T22:57:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T22:57:01.871 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T22:57:01.873 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T22:57:01.878 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T22:57:01.884 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T22:57:01.885 INFO:teuthology.orchestra.run.smithi111.stderr: 94.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T22:57:01.886 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% 94.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T22:57:01.887 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T22:57:01.888 INFO:teuthology.orchestra.run.smithi111.stderr: 91.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T22:57:01.890 INFO:teuthology.orchestra.run.smithi186.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T22:57:01.894 INFO:teuthology.orchestra.run.smithi186.stderr: 94.6% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T22:57:01.901 INFO:teuthology.orchestra.run.smithi186.stderr: 94.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T22:57:01.903 INFO:teuthology.orchestra.run.smithi111.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T22:57:01.914 INFO:teuthology.orchestra.run.smithi186.stderr: 93.1% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T22:57:01.939 INFO:teuthology.orchestra.run.smithi111.stderr: 93.2% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T22:57:02.137 INFO:teuthology.orchestra.run.smithi171.stderr: 93.8% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T22:57:02.177 INFO:teuthology.orchestra.run.smithi171.stderr: 91.8% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T22:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr: 90.1% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T22:57:02.404 INFO:teuthology.orchestra.run.smithi111.stderr: 92.7% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T22:57:02.407 INFO:teuthology.orchestra.run.smithi186.stderr: 91.7% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T22:57:02.444 INFO:teuthology.orchestra.run.smithi186.stderr: 92.9% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T22:57:02.458 INFO:teuthology.orchestra.run.smithi186.stderr: 92.8% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T22:57:02.562 INFO:teuthology.orchestra.run.smithi171.stderr: 92.2% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T22:57:02.564 INFO:teuthology.orchestra.run.smithi186.stderr: 93.2% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T22:57:02.581 INFO:teuthology.orchestra.run.smithi171.stderr: 92.9% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T22:57:02.645 INFO:teuthology.orchestra.run.smithi111.stderr: 92.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T22:57:02.680 INFO:teuthology.orchestra.run.smithi171.stderr: 93.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T22:57:02.728 INFO:teuthology.orchestra.run.smithi111.stderr: 93.1% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T22:57:02.794 INFO:teuthology.orchestra.run.smithi111.stderr: 93.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T22:57:02.802 INFO:teuthology.orchestra.run.smithi111.stderr: 90.5% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T22:57:02.805 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-04-24T22:57:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m0.977s 2024-04-24T22:57:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m4.266s 2024-04-24T22:57:02.805 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.419s 2024-04-24T22:57:02.877 INFO:teuthology.orchestra.run.smithi186.stderr: 93.0% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T22:57:02.878 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-24T22:57:02.879 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m1.065s 2024-04-24T22:57:02.879 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m3.711s 2024-04-24T22:57:02.879 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.143s 2024-04-24T22:57:02.902 INFO:teuthology.orchestra.run.smithi171.stderr: 93.3% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T22:57:03.793 INFO:teuthology.orchestra.run.smithi171.stderr: 92.8% -- replaced with /var/log/ceph/0d3ba9b8-028a-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T22:57:03.795 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-24T22:57:03.795 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m1.981s 2024-04-24T22:57:03.795 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m5.850s 2024-04-24T22:57:03.795 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.228s 2024-04-24T22:57:03.795 INFO:tasks.cephadm:Archiving logs... 2024-04-24T22:57:03.795 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi111/log 2024-04-24T22:57:03.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T22:57:04.480 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi171/log 2024-04-24T22:57:04.481 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T22:57:05.320 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi186/log 2024-04-24T22:57:05.322 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T22:57:05.890 INFO:tasks.cephadm:Removing cluster... 2024-04-24T22:57:05.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force 2024-04-24T22:57:06.204 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:57:06.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force 2024-04-24T22:57:06.961 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:57:07.222 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d3ba9b8-028a-11ef-bc93-c7b262605968 --force 2024-04-24T22:57:07.413 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0d3ba9b8-028a-11ef-bc93-c7b262605968 2024-04-24T22:57:07.680 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T22:57:07.680 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T22:57:07.738 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T22:57:07.753 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T22:57:07.769 INFO:tasks.cephadm:Teardown complete 2024-04-24T22:57:07.770 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T22:57:07.802 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T22:57:07.802 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T22:57:07.811 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T22:57:07.813 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T22:57:07.960 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T22:57:08.076 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T22:57:08.076 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:57:08.086 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:57:08.090 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:57:08.101 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:57:08.104 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:^- hv01.front.sepia.ceph.com 2 6 377 41 +1267us[+1270us] +/- 574ms 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -44us[ -42us] +/- 71ms 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +52us[ +54us] +/- 64ms 2024-04-24T22:57:08.107 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:57:08.108 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:57:08.108 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-24T22:57:08.108 INFO:teuthology.orchestra.run.smithi186.stdout:^- hv01.front.sepia.ceph.com 2 6 377 40 +1262us[+1262us] +/- 574ms 2024-04-24T22:57:08.108 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -45us[ -49us] +/- 71ms 2024-04-24T22:57:08.108 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +51us[ +51us] +/- 64ms 2024-04-24T22:57:08.109 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:57:08.132 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:^- hv01.front.sepia.ceph.com 2 7 377 104 +1350us[+1404us] +/- 573ms 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -43us[ +20us] +/- 71ms 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +57us[ +57us] +/- 64ms 2024-04-24T22:57:08.141 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:57:08.142 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T22:57:08.328 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T22:57:08.328 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T22:57:08.415 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T22:57:08.611 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T22:57:08.680 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T22:57:08.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:08.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:08.847 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:08.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:08.879 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:08.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:08.909 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T22:57:08.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T22:57:08.935 INFO:teuthology.task.internal:Duration was 2163.937264 seconds 2024-04-24T22:57:08.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T22:57:08.952 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T22:57:08.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T22:57:08.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T22:57:08.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T22:57:08.999 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:09.000 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:09.098 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:09.445 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T22:57:09.445 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-04-24T22:57:09.446 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T22:57:09.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-24T22:57:09.522 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T22:57:09.545 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-24T22:57:09.546 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T22:57:09.577 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T22:57:09.577 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T22:57:09.586 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T22:57:09.590 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T22:57:09.664 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T22:57:09.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T22:57:09.670 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T22:57:09.672 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T22:57:10.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T22:57:10.170 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T22:57:10.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T22:57:10.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T22:57:10.273 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T22:57:10.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T22:57:10.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.318 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.321 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.341 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-24T22:57:10.359 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-24T22:57:10.407 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-04-24T22:57:10.448 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:10.494 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:10.509 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:57:10.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:10.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T22:57:10.536 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T22:57:10.537 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi111 2024-04-24T22:57:10.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T22:57:10.643 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi171 2024-04-24T22:57:10.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T22:57:10.683 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672127/remote/smithi186 2024-04-24T22:57:10.683 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T22:57:10.719 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T22:57:10.719 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:10.726 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:10.729 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:10.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T22:57:10.792 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T22:57:10.792 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T22:57:10.804 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T22:57:10.804 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T22:57:10.809 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T22:57:10.811 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T22:57:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 22:57 /home/ubuntu/cephtest 2024-04-24T22:57:10.825 INFO:teuthology.orchestra.run.smithi186.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 22:57 /home/ubuntu/cephtest 2024-04-24T22:57:10.857 INFO:teuthology.orchestra.run.smithi111.stdout: 393584 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 22:57 /home/ubuntu/cephtest 2024-04-24T22:57:10.859 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T22:57:10.872 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T22:57:10.900 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 2163.937263727188 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T22:57:10.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:57:11.012 INFO:teuthology.run:pass